-- Logs begin at Tue 2022-09-13 09:26:13 UTC, end at Tue 2022-09-13 10:33:09 UTC. -- Sep 13 09:56:14.876596 dv-cinder-854274-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 09:56:14.885212 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 09:56:14 2022] *** Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: nodename: dv-cinder-854274-4-iscsi Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: machine: x86_64 Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: clock source: unix Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: pcre jit disabled Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: detected number of CPU cores: 4 Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: current working directory: / Sep 13 09:56:14.887255 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: detected binary path: /usr/bin/uwsgi-core Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: your processes number limit is 47680 Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: your memory page size is 4096 bytes Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: detected max file descriptor number: 2048 Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: lock engine: pthread robust mutexes Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: thunder lock: enabled Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 09:56:14.887809 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 09:56:14.929052 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: Python main interpreter initialized at 0x55a93fb659c0 Sep 13 09:56:14.929052 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: python threads support enabled Sep 13 09:56:14.929052 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: your server socket listen backlog is limited to 100 connections Sep 13 09:56:14.929052 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: your mercy for graceful operations on workers is 90 seconds Sep 13 09:56:14.929418 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 09:56:14.929502 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: *** Operational MODE: preforking *** Sep 13 09:56:14.929735 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: *** uWSGI is running in multiple interpreter mode *** Sep 13 09:56:14.929735 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: spawned uWSGI master process (pid: 81724) Sep 13 09:56:14.929939 dv-cinder-854274-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 09:56:14.933193 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: spawned uWSGI worker 1 (pid: 81736, cores: 1) Sep 13 09:56:14.933193 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: spawned uWSGI worker 2 (pid: 81737, cores: 1) Sep 13 09:56:14.933193 dv-cinder-854274-4-iscsi devstack@keystone.service[81724]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 09:56:19.288350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 09:56:19.288350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: from cryptography.utils import int_from_bytes Sep 13 09:56:19.289120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 09:56:19.289120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: from cryptography.utils import int_from_bytes Sep 13 09:56:19.293085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 09:56:19.293085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: from cryptography.utils import int_from_bytes Sep 13 09:56:19.293534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 09:56:19.293534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: from cryptography.utils import int_from_bytes Sep 13 09:56:19.344075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 09:56:19.344272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 09:56:19.344488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] command line args: [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 09:56:19.344603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 09:56:19.344682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 09:56:19.344752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 09:56:19.344884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] command line args: [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 09:56:19.344962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ================================================================================ {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 09:56:19.345146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] admin_token = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.345228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 09:56:19.345328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ================================================================================ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 09:56:19.345410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] config_dir = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.345571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.345647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] admin_token = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.345757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] config_source = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.345840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config_dir = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.345941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] debug = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config_source = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] debug = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.346980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_config_append = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_config_append = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.347950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_dir = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_file = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_dir = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.348940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.349936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.350970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.351929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] publish_errors = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] publish_errors = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.352996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] standard_threads = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.353991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] standard_threads = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] use_journal = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] use_json = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_journal = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] use_stderr = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.354914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_json = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.355012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] use_syslog = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.355087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_stderr = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.355249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_syslog = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.355327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.355486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 09:56:19.355564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.355745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.355821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.355985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.356959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.357926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.358964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.359900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.360994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.361848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.362964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.363779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.364465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.364548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.364716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.364798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.364948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.365022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.365186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.365262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.365419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.365495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.365708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.366997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.367963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.368979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.369911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.370851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.371971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.372939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.373886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.374904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.375071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.375145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.377979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.378957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.379578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.380967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.381047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.381119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.381193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.381263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.external = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.382943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.external = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.password = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] auth.token = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.383909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.password = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.token = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.384938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.385914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386107 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.386985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.387984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.388929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.389827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.390009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.390083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.390261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.390345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.390495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.391949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.392945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.393907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.394894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.395904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.396862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.397983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.398948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.399937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.400929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401162 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.401947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.402934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.403893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404824 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.404926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.405874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.406996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.407974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.408990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.409953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.410924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.411896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.412873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.413971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.414927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.415015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.415205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.415285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] role.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.415433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.415511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] role.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.415921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.416974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.417941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.418917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.419930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420755 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.420993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.421905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.422887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.423890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.424038 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.424115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.424272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.424340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.424518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.424584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.425981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.426989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.427947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.428923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.connection = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.429884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430519 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.430840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.431980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.432992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.433153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.433229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.433325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81736) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 09:56:19.433410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.433627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.433875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.434112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.434358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.434590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.434832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.435068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 09:56:19.435263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 09:56:19.462865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.463094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.463351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.463918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.464156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.464387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.471627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.472857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.502918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.503145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.503304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.503464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.570072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.570259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.618089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.618323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.619321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.619562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.781971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.782109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.782191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.782331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.815730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.815963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.816921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.817071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.817151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.817291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.817508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.828399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.828515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.846971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.847915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.848006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.848132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.848215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.848863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81736) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.848939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 09:56:19.887591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.887846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 09:56:19.888135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.888272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.888411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 09:56:19.888541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 09:56:19.888691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.888827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.888959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.889089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 09:56:19.889232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.889360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.889495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.889622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.889622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.889863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.899021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.899328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.899606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.899897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.900027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.900199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.900471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.900754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.906921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.907793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.909850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.910139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.910427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.910686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.911329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.911488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.911628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.911768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.911923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.912900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.913100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.913200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.913365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.913638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.913904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.936667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.958468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.958579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.958579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.958723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.958723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.958875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.958875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.958875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.959070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.959308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.959444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.959574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.959786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.959963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.960462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.974017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.974301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.974562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.974840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.978195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.978741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.981419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.981721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.982019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.982331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.985920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.986562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.989700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.990367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:19.992972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.993272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.993567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.993664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.993854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.993959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.994143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.994245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.994434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.994525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.994806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:19.995092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.010399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.010692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.010963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.011043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.011258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.011357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.011589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.011854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.018172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.018713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.019741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.020298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.021299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.021593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.021847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.022124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.024360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.024645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.024862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.024945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.025147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.025244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.025429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.025587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.025709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.026115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.037379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.037666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.037839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.037952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.038122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.038239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.038400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.038621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.038702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.038912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.039039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.039218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.039347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.039649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.055644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.055862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.058561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.058781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.061488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.061707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.063015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.063152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.064528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.064639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.064814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.064932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.065220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.065311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.071739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.071829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.073182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.073288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.076114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.076201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.076507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.076589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.076791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.076885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.077076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.077158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.085523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.085620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.085774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.085857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.086054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.086136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.093000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.093096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81736) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 09:56:20.093284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.093369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.093563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.093645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.093840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.093931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.094942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.095863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.096056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.096145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.096330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.096410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.096597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.096677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81736) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 09:56:20.131304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a93fb659c0 pid: 81736 (default app) Sep 13 09:56:20.131413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a93fb659c0 pid: 81737 (default app) Sep 13 09:56:20.134585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699d39-760f-4d5a-bf84-4442352f4d08 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:20.134876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699d39-760f-4d5a-bf84-4442352f4d08 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:20.135152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699d39-760f-4d5a-bf84-4442352f4d08 None None] PATH_INFO: `/v3` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:20.139368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Tue Sep 13 09:56:20 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:27.739562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:27.740049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:27.740130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:27.744342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:56:27.749408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:27.749806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:27.749963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:27.822016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: INFO dbcounter [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] Registered counter for database keystone Sep 13 09:56:27.840708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG oslo_db.sqlalchemy.engines [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81736) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 09:56:27.846959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writer thread running {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 09:56:28.026431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG passlib.handlers.bcrypt [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81736) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 09:56:28.026752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG passlib.handlers.bcrypt [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81736) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 09:56:28.085854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:28.087503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:28.217675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 09:56:28.654690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:28.725680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: INFO dbcounter [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] Registered counter for database keystone Sep 13 09:56:28.746651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG oslo_db.sqlalchemy.engines [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81737) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 09:56:28.752831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writer thread running {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 09:56:28.872177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d542ab2-a13e-4579-bdde-bb6ab726616b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:28.876483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:28.877111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:28.877396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:28.877670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:28.886380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:28.937842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 09:56:28.938142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f25e9-3bb2-48d3-b5e7-c306dc0bb30d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:28.948078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Tue Sep 13 09:56:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 09:56:28.953979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:28.972019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63029125-c4ea-4c7e-a8f3-9c62148ec8da None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:28.976422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:28.977548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:28.977835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:28.978113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] PATH_INFO: `/v3/projects/60981a9be15c4b04b78b97afc4285aa0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:28.979741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] RBAC: Authorizing `identity:get_project(project_id=60981a9be15c4b04b78b97afc4285aa0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:29.032332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 09:56:29.032682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714c7ce-985e-41df-8b89-17f65cd909bd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:29.037209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Tue Sep 13 09:56:28 2022] GET /identity/v3/projects/60981a9be15c4b04b78b97afc4285aa0 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:31.138891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35454a18-aee8-41a7-b592-e7641006963a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:31.139439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35454a18-aee8-41a7-b592-e7641006963a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:31.139520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35454a18-aee8-41a7-b592-e7641006963a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:31.143682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:31.149176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:31.149654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:31.149744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:31.171053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:31.172576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:31.180897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 09:56:31.448071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35454a18-aee8-41a7-b592-e7641006963a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:31.460657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35454a18-aee8-41a7-b592-e7641006963a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:31.464928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cabc8f-fed0-4a47-b219-a9cd6fc29561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:31.465537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cabc8f-fed0-4a47-b219-a9cd6fc29561 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:31.465823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cabc8f-fed0-4a47-b219-a9cd6fc29561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:31.466096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cabc8f-fed0-4a47-b219-a9cd6fc29561 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:31.474297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cabc8f-fed0-4a47-b219-a9cd6fc29561 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:31.476117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cabc8f-fed0-4a47-b219-a9cd6fc29561 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:31.484819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Tue Sep 13 09:56:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 09:56:31.490064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:31.499109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3be784-ed7b-4229-96b2-e558f3253b75 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:31.503451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419919e3-02ce-44df-952b-cd3a8cf0b761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:31.504109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419919e3-02ce-44df-952b-cd3a8cf0b761 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:31.504393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419919e3-02ce-44df-952b-cd3a8cf0b761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:31.504670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419919e3-02ce-44df-952b-cd3a8cf0b761 admin admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:31.505983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419919e3-02ce-44df-952b-cd3a8cf0b761 admin admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:31.507827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419919e3-02ce-44df-952b-cd3a8cf0b761 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:31.509339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Tue Sep 13 09:56:31 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:33.694932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058efd19-b148-424b-a029-43d21196f31f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:33.695496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058efd19-b148-424b-a029-43d21196f31f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:33.695496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058efd19-b148-424b-a029-43d21196f31f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:33.696763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:33.702028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:33.702446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:33.702584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:33.720350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:33.721544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:33.766432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f5e2e-674b-4184-abfa-f3dd199b29ef None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:33.766805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f5e2e-674b-4184-abfa-f3dd199b29ef None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:33.767151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f5e2e-674b-4184-abfa-f3dd199b29ef None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:33.768508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:56:33.773862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:33.774149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:33.774429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:33.803957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:33.816213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG passlib.handlers.bcrypt [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81737) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 09:56:33.817145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG passlib.handlers.bcrypt [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81737) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 09:56:33.865723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:33.867313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:33.963993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:34.072310 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.095849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.175517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3b313-b39c-4c7c-b7f8-01d7ce02b8ae None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.209258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.209914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.210197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.210475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.222513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:34.231699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:34.232766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:56:34.233597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:34.237783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.239010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.319449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.339429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931d064-0dd5-4bf3-8460-302d7addb689 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.340206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55a2fe-9c82-4708-b734-787c2df66f07 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.341806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.374857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.375505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.375794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.376067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.376258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.376746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:34.377050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.377157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.377400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.377675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.378450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.391420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:34.397659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.446390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.447935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.495419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640cf1a-eebc-4804-af84-41b93bf9d83c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.528694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.529510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.529968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.530239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.531411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.532859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.534418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 9/16] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:34.540379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.546080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 8/17] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:56:34.551798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.554453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.554651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.638939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0fff-e83e-4405-9743-fe620b755d3b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.653711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b85d-d735-4782-a7cd-4f32dc6fba6e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.672435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.673062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.673346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.673613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.674721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.676223 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.677767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:34.683533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.686491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.687378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.687657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.687923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.706465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.708697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:56:34.726666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:34.727750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:56:34.728537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:56:34.728537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:56:34.728537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:34.728537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:56:34.733522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:34.765030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.784414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb83df5-d5fb-4318-9127-65faf1fb978e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.791297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:34.818305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.818925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.819242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.819521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.820248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.domain.id=default, user.id=47dfc0e354ea47fc9cdf40b73108b8fe) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:34.820555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.822008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763d81-d4d4-44fb-9891-bdc83460cef7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.828546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/role_assignments?user.id=47dfc0e354ea47fc9cdf40b73108b8fe&scope.domain.id=default&role.id=435125566e954d20a3b8273b7c806b98 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:34.834735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dff62-9239-4ccd-b6a6-829ae55eda17 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:34.870401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:34.871024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:34.871340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:34.871620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:34.872276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:34.872582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:34.873989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f6bef-467e-4f9c-a7dd-f333c0ab94c0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:34.893851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:37.309541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4a4db-d1b9-4509-808a-4f2a822a94f6 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:37.310084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4a4db-d1b9-4509-808a-4f2a822a94f6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:37.310160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4a4db-d1b9-4509-808a-4f2a822a94f6 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:37.311493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:37.316972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c03e2-f983-498b-a714-90700bafa0e6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:37.317409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c03e2-f983-498b-a714-90700bafa0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:37.317579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c03e2-f983-498b-a714-90700bafa0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:37.343259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c67c03e2-f983-498b-a714-90700bafa0e6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:37.344491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c67c03e2-f983-498b-a714-90700bafa0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:37.442485 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:37.858208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60a909-f7f0-4423-a9f2-aef95f602048 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:37.858602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60a909-f7f0-4423-a9f2-aef95f602048 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:37.858800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60a909-f7f0-4423-a9f2-aef95f602048 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:37.860229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 13/24] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:37.866179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:37.866469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:37.866736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:37.896194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5afe7d-920e-46a4-b420-74f7b2f0e1d1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:37.896514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5afe7d-920e-46a4-b420-74f7b2f0e1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:37.896750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5afe7d-920e-46a4-b420-74f7b2f0e1d1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:37.898080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 14/25] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:56:37.902916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce053fec-e1ad-4c69-a161-9f9a6c09cae2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:37.903225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce053fec-e1ad-4c69-a161-9f9a6c09cae2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:37.903508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce053fec-e1ad-4c69-a161-9f9a6c09cae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:37.913343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:37.914552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:37.931911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ce053fec-e1ad-4c69-a161-9f9a6c09cae2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:37.933499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ce053fec-e1ad-4c69-a161-9f9a6c09cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:37.997037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:38.027984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 12/26] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:38.035186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:38.036040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 15/27] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:38.037354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eea0f1-f34b-42c4-b123-5a80788e6a31 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.037644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eea0f1-f34b-42c4-b123-5a80788e6a31 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.037919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eea0f1-f34b-42c4-b123-5a80788e6a31 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.039953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 16/28] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:38.054818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.055322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.055765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.094227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:38.096518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:38.233886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 17/29] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:38.249308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602cee8-dd1c-4952-b480-70fc3acc5331 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:38.295863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:38.296503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.296787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.297060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.315731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:38.322935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:38.324377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:38.325411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 13/30] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:56:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:38.331950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:38.340866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:38.370984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:38.455945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:38.520982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebfcc-092d-488b-b9e5-37c05030da64 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:38.573084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da42df8-f8d5-40e9-b9e5-b51bd0476fe8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:38.584099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:38.585109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.585591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.586076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.586988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:38.587634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:38.589232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:38.603316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 18/31] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:38.604523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:38.611467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:38.612295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.612745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.613218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.624365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:38.626173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:56:38.641971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:38.642873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:56:38.643893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:56:38.643893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:56:38.643893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:38.645629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 14/32] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:56:38.646578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:38.677859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:38.723252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca638-ef3e-44a1-9add-6e74f8b85cd8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:38.772628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:38.773204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.773492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.773771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.774447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:38.775874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:38.797337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:38.812026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c113-a5af-489d-af3d-83d8c9aeca00 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:38.859571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:38.860572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:38.861065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:38.861536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:38.862422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:38.864050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:38.987550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 19/33] 10.222.0.62 () {60 vars in 1131 bytes} [Tue Sep 13 09:56:38 2022] POST /identity/v3/roles => generated 219 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:38.989443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.188112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 15/34] 10.222.0.62 () {60 vars in 1131 bytes} [Tue Sep 13 09:56:38 2022] POST /identity/v3/roles => generated 225 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:39.189853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.190835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894fd0d-4ded-48fb-b874-92ffdf404c08 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.230411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:39.242159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f3002-374e-4ad0-b781-b35562e52fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:39.242829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f3002-374e-4ad0-b781-b35562e52fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.243116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f3002-374e-4ad0-b781-b35562e52fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.244710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f3002-374e-4ad0-b781-b35562e52fd2 None admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.244986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f3002-374e-4ad0-b781-b35562e52fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:39.246582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f3002-374e-4ad0-b781-b35562e52fd2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:39.248577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 20/35] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:38 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:39.253143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e0ee4-4bb8-4c5e-8a35-b848d3a25028 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.253143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e0ee4-4bb8-4c5e-8a35-b848d3a25028 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.253143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e0ee4-4bb8-4c5e-8a35-b848d3a25028 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.253143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 21/36] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:39.253517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c854be-687b-4e15-8a7f-ac096498ecb7 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.254893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c854be-687b-4e15-8a7f-ac096498ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.254893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c854be-687b-4e15-8a7f-ac096498ecb7 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.256710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 22/37] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:39.258157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c854be-687b-4e15-8a7f-ac096498ecb7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.337704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e1a6-5e9f-44f0-bd1d-3f3387bad231 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.377129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:39.377984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.378280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.378558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.380615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:39.380615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:39.381685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517e3e-2a2e-4905-83e5-777a83e73d94 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:39.390457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c854be-687b-4e15-8a7f-ac096498ecb7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.398390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 16/38] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:39.399468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.399765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.400047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.423231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:39.424829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:39.426843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:39.434542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d2b33-479e-4345-ad51-6dc6fac7cf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:39.434542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d2b33-479e-4345-ad51-6dc6fac7cf17 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.434542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d2b33-479e-4345-ad51-6dc6fac7cf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.434542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d2b33-479e-4345-ad51-6dc6fac7cf17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.436426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d2b33-479e-4345-ad51-6dc6fac7cf17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:39.437260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d2b33-479e-4345-ad51-6dc6fac7cf17 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:39.438880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 23/39] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:39.439995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.440285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.440565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.459918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:39.461541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:39.496166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 17/40] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:39.497010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.518808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:39.541981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 24/41] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:39.541981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.622876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565804cb-b9bd-4238-88bd-71c4ee04f225 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.663228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15383913-dea5-4f2f-9b3f-8e16031cb453 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.670474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:39.671639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.671928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.672208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.673645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:39.676328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:39.689781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:39.700335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:39.701171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.701449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.701759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] PATH_INFO: `/v3/domains/default/users/47dfc0e354ea47fc9cdf40b73108b8fe/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.703798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=47dfc0e354ea47fc9cdf40b73108b8fe, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:39.705236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:39.788431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 18/42] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:56:39 2022] POST /identity/v3/projects => generated 301 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:39.789256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.843196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 25/43] 10.222.0.62 () {58 vars in 1361 bytes} [Tue Sep 13 09:56:39 2022] PUT /identity/v3/domains/default/users/47dfc0e354ea47fc9cdf40b73108b8fe/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:56:39.843196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:39.924525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:39.933190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b7fd2-8402-43cb-a688-7d486d67b9aa None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.987701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4772c1-2a07-4709-a99f-988046a526ff None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:39.994827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c0e56-de96-4d9b-8e9c-a2e1cc814456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:39.995528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c0e56-de96-4d9b-8e9c-a2e1cc814456 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:39.995820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c0e56-de96-4d9b-8e9c-a2e1cc814456 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:39.996101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c0e56-de96-4d9b-8e9c-a2e1cc814456 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:39.996798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c0e56-de96-4d9b-8e9c-a2e1cc814456 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:39.998277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c0e56-de96-4d9b-8e9c-a2e1cc814456 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:40.031765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:40.032417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:40.032757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:40.033040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:40.035837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:40.037403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:40.039349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 26/44] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:40.045038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:40.092546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 19/45] 10.222.0.62 () {60 vars in 1131 bytes} [Tue Sep 13 09:56:39 2022] POST /identity/v3/roles => generated 223 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:40.163515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0f20-407d-4e70-a987-86574897febe None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:40.186943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:40.203778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5f714-933d-4f04-a271-3d18bea468ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:40.204399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f714-933d-4f04-a271-3d18bea468ce None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:40.204688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f714-933d-4f04-a271-3d18bea468ce None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:40.204971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f714-933d-4f04-a271-3d18bea468ce None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:40.205641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5f714-933d-4f04-a271-3d18bea468ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:40.207184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5f714-933d-4f04-a271-3d18bea468ce None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:40.419482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 27/46] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:56:40 2022] POST /identity/v3/projects => generated 312 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:56:42.035135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d5c6-4409-4ca0-a867-83eedfde1947 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.035669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d5c6-4409-4ca0-a867-83eedfde1947 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.035749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d5c6-4409-4ca0-a867-83eedfde1947 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.037072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 20/47] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:42.042731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.043143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.043338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.061848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:42.063047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:42.128441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 28/48] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:42.395975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d5c6-4409-4ca0-a867-83eedfde1947 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:42.451906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:42.507835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d5c6-4409-4ca0-a867-83eedfde1947 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:42.541904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d452f46-2335-443b-b796-c64f7e8a54f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:42.542551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d452f46-2335-443b-b796-c64f7e8a54f5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.542836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d452f46-2335-443b-b796-c64f7e8a54f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.543115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d452f46-2335-443b-b796-c64f7e8a54f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.553920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d452f46-2335-443b-b796-c64f7e8a54f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6d452f46-2335-443b-b796-c64f7e8a54f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:42.558831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:42.559707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:42.560584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 21/49] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:56:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:42.565789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:42.570663 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebacd9-7d9f-47f9-9357-b25a1036c829 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.570952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebacd9-7d9f-47f9-9357-b25a1036c829 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.571259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebacd9-7d9f-47f9-9357-b25a1036c829 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.572564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 22/50] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:56:42.577579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.577866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.578148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.596437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:42.598208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:42.667268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e394cfe-956c-4f87-b8c9-6cf1777ff92e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:42.668902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:42.703747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:42.704775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.705096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.705372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.706052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:42.706359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:42.708326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:42.717713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 29/51] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:42.724667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:42.768838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 23/52] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:42.822666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691669f-644d-4241-a99a-581b9d89bd65 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:42.855530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:42.856158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.856440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.856711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.857840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:42.859304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:42.865341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 30/53] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:42 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:42.866526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:42.907303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:42.963728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a74ad6-25d1-464d-a529-e235a1a946e6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:42.995209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:42.995843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:42.996127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:42.996404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:42.997506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:42.998964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:43.000565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 24/54] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:43.006765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:43.038355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:43.049203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:43.109947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca841c7-a770-4349-b714-3466ed73b800 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:43.143735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:43.144907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:43.145194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:43.145471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:43.146191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.domain.id=default, user.id=47dfc0e354ea47fc9cdf40b73108b8fe) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:43.146500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:43.147964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:43.150621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be595ff5-3493-41e9-9f0e-b4807b751f87 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:43.153774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 31/55] 10.222.0.62 () {56 vars in 1322 bytes} [Tue Sep 13 09:56:43 2022] GET /identity/v3/role_assignments?user.id=47dfc0e354ea47fc9cdf40b73108b8fe&scope.domain.id=default&role.id=435125566e954d20a3b8273b7c806b98 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:43.160013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:43.186196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e82e-1750-4857-ae31-4a146e08e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:43.186943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e82e-1750-4857-ae31-4a146e08e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:43.187264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e82e-1750-4857-ae31-4a146e08e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:43.187637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e82e-1750-4857-ae31-4a146e08e6d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:43.188823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e82e-1750-4857-ae31-4a146e08e6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:43.190297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e82e-1750-4857-ae31-4a146e08e6d3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:43.192514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 25/56] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:43.198358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:43.301683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:43.307518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc59cd-05e2-4c1d-a1b3-f3d76a24e894 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:43.345218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3ebfd-6584-4a8b-b14f-516847a0d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:43.345867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ebfd-6584-4a8b-b14f-516847a0d8c2 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:43.346159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ebfd-6584-4a8b-b14f-516847a0d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:43.346440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ebfd-6584-4a8b-b14f-516847a0d8c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:43.347120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3ebfd-6584-4a8b-b14f-516847a0d8c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:43.348628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3ebfd-6584-4a8b-b14f-516847a0d8c2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:43.434228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 32/57] 10.222.0.62 () {60 vars in 1140 bytes} [Tue Sep 13 09:56:43 2022] POST /identity/v3/projects => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:45.584492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c160c7-fe92-4502-9c4d-dae21ab500d8 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:45.585027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c160c7-fe92-4502-9c4d-dae21ab500d8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:45.585027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c160c7-fe92-4502-9c4d-dae21ab500d8 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:45.586402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 26/58] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:56:45.591661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:45.592014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:45.592228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:45.611833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:45.614337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:45.684606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 33/59] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:45.952410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c160c7-fe92-4502-9c4d-dae21ab500d8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:46.054336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c160c7-fe92-4502-9c4d-dae21ab500d8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:46.086743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312004d-8e5b-4c04-aead-3185ab6a5871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:46.087406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312004d-8e5b-4c04-aead-3185ab6a5871 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:46.087690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312004d-8e5b-4c04-aead-3185ab6a5871 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:46.087965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312004d-8e5b-4c04-aead-3185ab6a5871 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:46.089081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312004d-8e5b-4c04-aead-3185ab6a5871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:46.090511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312004d-8e5b-4c04-aead-3185ab6a5871 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:46.092108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 27/60] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:46.098191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:46.172853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:46.193784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a83959-eea6-45ed-b009-3d96c3a006d4 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:46.226827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f512-b861-4c36-84f5-f210a7b297cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:46.227400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f512-b861-4c36-84f5-f210a7b297cd None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:46.227688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f512-b861-4c36-84f5-f210a7b297cd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:46.227966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f512-b861-4c36-84f5-f210a7b297cd None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:46.228624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f512-b861-4c36-84f5-f210a7b297cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:46.230071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f512-b861-4c36-84f5-f210a7b297cd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:46.317933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 34/61] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:56:46 2022] POST /identity/v3/users => generated 275 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:49.472158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c8f6d-9c49-480e-84f1-3bf820dcf685 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:49.473037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c8f6d-9c49-480e-84f1-3bf820dcf685 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:49.473459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c8f6d-9c49-480e-84f1-3bf820dcf685 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:49.474960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 28/62] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:49.480264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485ce3-ad79-484c-8cee-3124e7dac260 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:49.481087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485ce3-ad79-484c-8cee-3124e7dac260 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:49.481514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485ce3-ad79-484c-8cee-3124e7dac260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:49.504825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-29485ce3-ad79-484c-8cee-3124e7dac260 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:49.506273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-29485ce3-ad79-484c-8cee-3124e7dac260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:49.597917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 35/63] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:49.783979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bfa20-7c2f-4d7c-bc19-431b46cb9d40 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:49.784294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bfa20-7c2f-4d7c-bc19-431b46cb9d40 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:49.785067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bfa20-7c2f-4d7c-bc19-431b46cb9d40 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:49.786587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 29/64] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:49.792399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e3958-43f4-40f7-9728-6b4a58675183 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:49.792695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e3958-43f4-40f7-9728-6b4a58675183 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:49.792982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e3958-43f4-40f7-9728-6b4a58675183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:49.820276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-350e3958-43f4-40f7-9728-6b4a58675183 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:49.821524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-350e3958-43f4-40f7-9728-6b4a58675183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:49.891843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:49.910259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 36/65] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:50.033477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bfa20-7c2f-4d7c-bc19-431b46cb9d40 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.097271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.156904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814004-658a-45b9-b6a3-e55b643eed58 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.157534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814004-658a-45b9-b6a3-e55b643eed58 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.158011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814004-658a-45b9-b6a3-e55b643eed58 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.159754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 37/66] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:56:50.165954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9e2a1-4691-4852-ace0-fd2b22596474 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.166600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9e2a1-4691-4852-ace0-fd2b22596474 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.167080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9e2a1-4691-4852-ace0-fd2b22596474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.179370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bfa20-7c2f-4d7c-bc19-431b46cb9d40 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.197639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-09c9e2a1-4691-4852-ace0-fd2b22596474 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:50.198869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-09c9e2a1-4691-4852-ace0-fd2b22596474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:50.230990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.231673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.232110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.232545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.244369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:50.249715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:50.250545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:50.251391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 30/67] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:50.252011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.279418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 38/68] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:50.280762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13af197-3ce6-441a-8e29-d69571e279be None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.281057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13af197-3ce6-441a-8e29-d69571e279be None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.281337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13af197-3ce6-441a-8e29-d69571e279be None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.282572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 39/69] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:50.283381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13af197-3ce6-441a-8e29-d69571e279be None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.375321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0587b4-42f5-4804-8b59-bab2c3587368 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.377664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.395946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13af197-3ce6-441a-8e29-d69571e279be None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.415027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c644066-2f65-4e20-a82f-e2e722567132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.415948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c644066-2f65-4e20-a82f-e2e722567132 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.416236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c644066-2f65-4e20-a82f-e2e722567132 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.416509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c644066-2f65-4e20-a82f-e2e722567132 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.421221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.427439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c644066-2f65-4e20-a82f-e2e722567132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1c644066-2f65-4e20-a82f-e2e722567132 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:50.432284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:50.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:50.433836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 31/70] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:50.434584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d4d2f-d79a-4a35-854b-1dccd60a09d6 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.435023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d4d2f-d79a-4a35-854b-1dccd60a09d6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.435331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d4d2f-d79a-4a35-854b-1dccd60a09d6 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.441517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 32/71] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:50.441517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267f33-7fa5-4233-aaa8-bd00667fe60f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.441517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267f33-7fa5-4233-aaa8-bd00667fe60f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.441517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267f33-7fa5-4233-aaa8-bd00667fe60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.441823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.441823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.441823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.441823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.442088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:50.442088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:50.443587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:50.458732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 40/72] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:50.458732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.458960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b0267f33-7fa5-4233-aaa8-bd00667fe60f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:50.460192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b0267f33-7fa5-4233-aaa8-bd00667fe60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:50.530367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 33/73] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:50.531686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.531977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.532254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.551207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:50.552419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:50.580869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a1fde-1ad6-4262-af68-a409413415b3 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.616939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.617953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.618530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.619026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.619934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:50.620105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 34/74] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:50.620594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:50.620929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.622434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:50.638664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 41/75] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:50.640044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.666509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.703551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.745468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbd436-36f9-4c49-baab-75abeb5db63d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.758713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1413c-66c7-4f75-8e74-a909d79117d9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.781391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.782033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.782320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.782598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.798170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.798971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:50.799358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.799889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.800397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.800515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:50.802343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 35/76] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:50.807441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.812906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:50.819939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:50.820790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:56:50.821609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 42/77] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:50.822213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.897710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.917188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4326c6-ddae-4cfd-84ad-8b18446c54cc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.931896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee967f-ed1a-4427-b24d-995c8b641ff4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:50.940629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:50.952198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.952830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.953116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.953393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:50.954524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:50.955993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:50.957678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 36/78] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:50.958438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:50.965708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:50.966331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:50.966621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:50.966899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.042315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.043840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.067795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39551fbc-3ed1-4847-bbd5-61f2c5305b00 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.103848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.104478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.104761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.105035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.106143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.107616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.109168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 37/79] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:51.114315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.132915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.141163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 43/80] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:56:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:56:51.142136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.152308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.211868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786df31-a510-4bd0-a4b3-3ff45a7d67db None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.240625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06d049-c186-4b06-9ea6-a4e2879575a5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.245104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.245724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.245993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.246268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.246926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:51.247258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.248629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.264521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 38/81] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:51.264521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.274287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.274928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.275255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.275524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.286024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:51.291137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:51.291969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:56:51.292727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 44/82] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:51.293360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.352033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.366184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.370854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6152ca-cbc9-46cd-bf10-a528540a756b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.399256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cf86-3bdb-456c-8e51-763b45ea642b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.405596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.406154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.406441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.406715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.434901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.435541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.435831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.436107 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.436751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.438217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.473448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.474907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.546493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 45/83] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:56:51 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:56:51.550954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.564270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.572080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 39/84] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:56:51.573226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.688080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1329b2-b919-4605-a4fc-9c9d5f5eab65 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.693045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfbc7-cf62-4346-bac9-4fc1ee85db9c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.720915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.729494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.735556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.735556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.735556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.735556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.735556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.735556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 40/85] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:51.735995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.736085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.736252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.736674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.737210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.740369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.742702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.744752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 46/86] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:51.745671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.802102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:51.871204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8938ed-2b4e-4b64-abf5-8476c63a91ff None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.874439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba796c5c-d5bf-4ace-9336-4c606934375b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:51.917348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.917868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:51.918006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.918241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.918526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.919220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:51.919472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:51.919586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.920043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:51.920557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:51.920924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.922091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:51.924321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:51.926209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 47/87] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:51.927314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.930115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 41/88] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:51.931046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:51.992890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.038009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a104936-c686-40ef-889a-cc1b1c1d34b8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.045673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ffa13-3d69-47b1-ab5c-78e77691af74 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.057105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.077421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.078327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.078827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.079338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.084564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.085224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.085510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.085837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.086575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=a736dfa50d3841bb971419573615ad37) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:52.086885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.088369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.098755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 42/89] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/role_assignments?user.id=a736dfa50d3841bb971419573615ad37&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:52.099506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:52.160010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.161990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.214387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eec8aa-7298-431b-963f-49bc1f79334f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.255280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.255477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.255555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.255555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.255555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.255555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=47dfc0e354ea47fc9cdf40b73108b8fe) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:52.255555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.258907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.264655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 43/90] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:56:52 2022] GET /identity/v3/role_assignments?user.id=47dfc0e354ea47fc9cdf40b73108b8fe&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:52.264655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:52.312939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.343595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 48/91] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:56:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:56:52.351124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:52.384221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed383cc-1bde-4638-a836-c20facc227d9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.423697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.424842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.425673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.425952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.427231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.428748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.431375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 44/92] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:52 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:52.435902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:52.541621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7adc3f-cd3c-40a3-8100-15a0ed48ca74 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.559701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.578897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.579627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.583771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.584310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.615489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a348-ba24-45ce-94db-0e9c2b754606 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.683059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.684190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.685305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.685935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.688496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.692179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.694248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 49/93] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:52 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:52.703194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.703194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.704532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:52.755538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.861460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 45/94] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:56:52.866580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719fbfd-3d55-44c9-8282-1baa4b46c74c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:52.867575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:52.880969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:52.912005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:52.912630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:52.912919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:52.913199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:52.915753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] RBAC: Adding query filter params (role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=a736dfa50d3841bb971419573615ad37) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:52.916000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:52.917463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:52.935155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 50/95] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:56:52 2022] GET /identity/v3/role_assignments?user.id=a736dfa50d3841bb971419573615ad37&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:53.027741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685309e-1a37-4179-bd75-47597573f9df None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:53.065621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afae8f-1cb2-4c66-9ded-5243aa528803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:53.066925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afae8f-1cb2-4c66-9ded-5243aa528803 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:53.067237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afae8f-1cb2-4c66-9ded-5243aa528803 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:53.067521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afae8f-1cb2-4c66-9ded-5243aa528803 None admin] PATH_INFO: `/v3/projects/ad00bd6c4e9d494096a9749b38341af2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:53.068711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afae8f-1cb2-4c66-9ded-5243aa528803 None admin] RBAC: Authorizing `identity:get_project(project_id=ad00bd6c4e9d494096a9749b38341af2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:53.070218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afae8f-1cb2-4c66-9ded-5243aa528803 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:53.072237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 46/96] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:52 2022] GET /identity/v3/projects/ad00bd6c4e9d494096a9749b38341af2 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:53.078815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:53.182797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:53.197752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd374-8f46-4ebb-8d9b-21220eadc9c5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:53.240550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:53.241186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:53.241476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:53.241760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:53.242577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=ad00bd6c4e9d494096a9749b38341af2, user.id=a736dfa50d3841bb971419573615ad37) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:53.242902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:53.244414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a204f8-c364-4b39-aa13-38b35aa26e5c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:53.254883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 51/97] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:56:53 2022] GET /identity/v3/role_assignments?user.id=a736dfa50d3841bb971419573615ad37&scope.project.id=ad00bd6c4e9d494096a9749b38341af2&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:54.783012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f9c1e-4578-45e4-a0f8-0777f9c54747 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:54.783556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f9c1e-4578-45e4-a0f8-0777f9c54747 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:54.783628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f9c1e-4578-45e4-a0f8-0777f9c54747 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:54.785345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 47/98] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:54.790922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdc44c-181b-4988-a61c-aa0d82a13685 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:54.791354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdc44c-181b-4988-a61c-aa0d82a13685 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:54.791517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdc44c-181b-4988-a61c-aa0d82a13685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:54.816786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-06bdc44c-181b-4988-a61c-aa0d82a13685 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:54.818151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-06bdc44c-181b-4988-a61c-aa0d82a13685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:54.906568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 52/99] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:55.002294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437af79-4fa2-439a-9a6c-cd7b96585a19 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.002619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437af79-4fa2-439a-9a6c-cd7b96585a19 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.002862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437af79-4fa2-439a-9a6c-cd7b96585a19 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.004526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 48/100] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:55.009974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0bdeb-be6b-4b88-90a5-da85736d8eb7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.010266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0bdeb-be6b-4b88-90a5-da85736d8eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.010548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0bdeb-be6b-4b88-90a5-da85736d8eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.033135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-97f0bdeb-be6b-4b88-90a5-da85736d8eb7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:55.034280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-97f0bdeb-be6b-4b88-90a5-da85736d8eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:55.123679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 53/101] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:55.292473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437af79-4fa2-439a-9a6c-cd7b96585a19 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:55.399031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:55.443581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437af79-4fa2-439a-9a6c-cd7b96585a19 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:55.453796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14f6f-36c4-45b2-8a3c-262929826465 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.454368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14f6f-36c4-45b2-8a3c-262929826465 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.454853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14f6f-36c4-45b2-8a3c-262929826465 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.457006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 54/102] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:55.462519 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1a6f-a18c-4ef7-80fd-756c513d1c4e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.463083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1a6f-a18c-4ef7-80fd-756c513d1c4e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.463621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1a6f-a18c-4ef7-80fd-756c513d1c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.487228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3fde1a6f-a18c-4ef7-80fd-756c513d1c4e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:55.489546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3fde1a6f-a18c-4ef7-80fd-756c513d1c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:55.505175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:55.505386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:55.505466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.505466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.505466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.506492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:55.509586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:55.512151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 49/103] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:55.517684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:55.609751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 55/104] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:56:55.611482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4ffef-cc89-4434-98a6-6f3b3f54dec0 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.611912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4ffef-cc89-4434-98a6-6f3b3f54dec0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.612353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4ffef-cc89-4434-98a6-6f3b3f54dec0 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.613991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 56/105] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:55.614933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4ffef-cc89-4434-98a6-6f3b3f54dec0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:55.700508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e27e9-e7c0-4096-88a4-cdc1f7891c36 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:55.779409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:55.780037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.780325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.780603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.781328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:55.783610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:55.818045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:55.818932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4ffef-cc89-4434-98a6-6f3b3f54dec0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:55.855481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff5186-d5bd-46fb-97de-38e1c910a638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:55.856129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff5186-d5bd-46fb-97de-38e1c910a638 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.856463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff5186-d5bd-46fb-97de-38e1c910a638 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.856742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff5186-d5bd-46fb-97de-38e1c910a638 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff5186-d5bd-46fb-97de-38e1c910a638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-55ff5186-d5bd-46fb-97de-38e1c910a638 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:55.874132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:55.874898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:56:55.876319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 57/106] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:55.877033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:55.877033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:55.877145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:55.890968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:55.901406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:55.902616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:55.911120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 50/107] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:56:55 2022] POST /identity/v3/projects => generated 302 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:56:55.912160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.038969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 58/108] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:56.041558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.111290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef764578-9253-4f53-ac48-91e128f3f3eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.186851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.188273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.189083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.189660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.190555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:56.191627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:56.193329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc82e1-f62b-4f28-84c4-79b52dcc2da4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:56.205873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 51/109] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:56:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:56.207391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2965fb-d621-4679-ac06-f1f61a9a7c17 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.207673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2965fb-d621-4679-ac06-f1f61a9a7c17 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.207961 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2965fb-d621-4679-ac06-f1f61a9a7c17 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.210694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 52/110] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:56.211798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2965fb-d621-4679-ac06-f1f61a9a7c17 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.227444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51737421-aa56-4c91-a1e9-04304f512870 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.272367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0def1-bf16-4a00-b1ee-1c36ac0a6e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.273019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0def1-bf16-4a00-b1ee-1c36ac0a6e96 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.273299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0def1-bf16-4a00-b1ee-1c36ac0a6e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.273577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0def1-bf16-4a00-b1ee-1c36ac0a6e96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.277563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:56.288457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0def1-bf16-4a00-b1ee-1c36ac0a6e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-1dc0def1-bf16-4a00-b1ee-1c36ac0a6e96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:56.295859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:56.304300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:56.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:56:56.305646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:56.305713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 59/111] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:56.305713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.305713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.305713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.325225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:56.326421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:56.358379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2965fb-d621-4679-ac06-f1f61a9a7c17 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.405099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.406158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.406468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.406785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] PATH_INFO: `/v3/users/3502a3cecf6f4e9aac93ab50c72d986b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.424736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] RBAC: Authorizing `identity:get_user(user_id=3502a3cecf6f4e9aac93ab50c72d986b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:56.428338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:56.430138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 60/112] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:56.431003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.435584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 53/113] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/users/3502a3cecf6f4e9aac93ab50c72d986b => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:56.436484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.602465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d66f9-03d0-4811-9efc-c026b964d84b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.610461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6198a-fc73-44b5-8e20-46ec75bf71ea None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.618428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:56.639486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.640624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.640923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.641207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.648373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:56.654898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:56.658932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.659623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.659912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.660191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:56.660646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:56.662529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:56:56.664363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:56.666134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:56.666134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:56.666389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 54/114] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:56.666519 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dd61a-d7d8-498a-ba6d-1f6243d78138 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:56.666650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.679753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 61/115] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:56.681027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cdf4c-6a91-49d6-b51a-2ac3a21d064b None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.681320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cdf4c-6a91-49d6-b51a-2ac3a21d064b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.681605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cdf4c-6a91-49d6-b51a-2ac3a21d064b None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.682878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 62/116] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:56.683623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cdf4c-6a91-49d6-b51a-2ac3a21d064b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.786940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50e73-c076-42be-8803-a6aca89e6d02 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.797326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cdf4c-6a91-49d6-b51a-2ac3a21d064b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:56.823298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58218-0364-4de9-aec5-565465c3631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.823913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58218-0364-4de9-aec5-565465c3631b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.824199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58218-0364-4de9-aec5-565465c3631b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.824474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58218-0364-4de9-aec5-565465c3631b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.834034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:56.834655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:56.834939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:56.835245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:56.835931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:56.836240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:56.837614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:56.846948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 63/117] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:56.852334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:56.869387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:56.896675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d58218-0364-4de9-aec5-565465c3631b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:56.898170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d58218-0364-4de9-aec5-565465c3631b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:56.910037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:56.964183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7bc1-e27e-4410-ac13-e74aa054cf3f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.000829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.001472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.001805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.002082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.003912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.005352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.010998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 55/118] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:56:57.011140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 64/119] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:56 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.011439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.011512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.011576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.011576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.026359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:57.027516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:57.099888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 56/120] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:57.111403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.113397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.133550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ef7a-be43-41d8-97fa-3e637691b4e4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.156872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.179422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.180343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.180632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.180910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.195074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:57.201852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:57.202709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:56:57.203512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:56:57.207585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 65/121] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:57.208236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.230172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702210a1-0136-4b22-9de1-7f2e3d11dc30 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.270028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.270664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.271054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.271373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.272614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.274063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.275832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 57/122] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.276596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.334949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16bd3-4246-4a45-a0bc-64beb2abe5cb None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.359400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.379520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.380166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.380852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.381144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.385474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.388262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.391621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 66/123] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.395941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.405158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b1dc7-1e22-4cf4-ba91-5833c0abd1f1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.440702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.441345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.441629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.441905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.443051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.444539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.446346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 58/124] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.447069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.450217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.508057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864fa0-fd34-4123-b363-9e8fec5e4258 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.545106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.545913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.546332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.546562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ede42-591d-4e34-94f5-aadfe23c747f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.546921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.547757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:57.548249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.549939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.558598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 67/125] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.559683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.586942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.587944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.588403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.588685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.589818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.591285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.597497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 59/126] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.597497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.606404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.672724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4e854-b340-46ac-8418-2de2f06ba5ae None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.681880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.698305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacdd78-1294-4f10-9530-5aa901eecc05 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.708809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.709621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.709906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.710185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.710961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] RBAC: Adding query filter params (role.id=c099a4b10af549a6ab8cff946dca2e5f, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=3502a3cecf6f4e9aac93ab50c72d986b) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:57.711451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.713476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.731525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 68/127] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/role_assignments?user.id=3502a3cecf6f4e9aac93ab50c72d986b&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=c099a4b10af549a6ab8cff946dca2e5f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.732431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.733019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.733675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.734002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.734269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.756312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:56:57.764143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:56:57.765230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:56:57.766180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:56:57.768517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 60/128] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:56:57.771875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:57.855564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:57.883715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ac763-b830-4069-bbe0-7a1c505c8034 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.894814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6de979-c90e-427b-af05-069874886d8c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:57.927633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.928592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.928981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.929361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/a736dfa50d3841bb971419573615ad37/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.931661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a44c7b4a430429e94e14dfb7e2c81a5, user_id=a736dfa50d3841bb971419573615ad37, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.933378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.933856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:57.934657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:57.935075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:57.935501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:57.936774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:57.938467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:57.940371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 61/129] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:57 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:57.941882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.007281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 69/130] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:56:57 2022] PUT /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/a736dfa50d3841bb971419573615ad37/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:56:58.008834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.076667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:58.104139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79447215-9f5b-469d-8949-95dc63540df0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:58.176580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:58.177316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.177710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.177932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/47dfc0e354ea47fc9cdf40b73108b8fe/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.179936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] RBAC: Authorizing `identity:create_grant(project_id=5a44c7b4a430429e94e14dfb7e2c81a5, user_id=47dfc0e354ea47fc9cdf40b73108b8fe, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:58.181417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:58.191216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:58.205593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aaefe-1b9c-4757-a895-0377cd3fc274 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:58.245437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:58.246306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.246728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.247141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.248037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:56:58.248505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:58.250010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:58.259812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 70/131] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:56:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:58.260822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.286584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 62/132] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:56:57 2022] PUT /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/47dfc0e354ea47fc9cdf40b73108b8fe/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:56:58.287918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.451835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:58.466129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d2ad3-442f-4b2c-b2fa-8d0fce02f7a5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:58.514999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82c7c-1f03-4cd8-b856-6e5f0c4fb2ba None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:58.533902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba695147-25e7-4057-a7d6-816f7196610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:58.535179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba695147-25e7-4057-a7d6-816f7196610a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.535474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba695147-25e7-4057-a7d6-816f7196610a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.535754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba695147-25e7-4057-a7d6-816f7196610a None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.537636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba695147-25e7-4057-a7d6-816f7196610a None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:58.539106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba695147-25e7-4057-a7d6-816f7196610a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:58.541226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 71/133] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:58 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:58.542217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a79a70-f727-40e1-9dcd-4d959496af42 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.542507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a79a70-f727-40e1-9dcd-4d959496af42 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.542783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a79a70-f727-40e1-9dcd-4d959496af42 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.546030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 72/134] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:56:58.554351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a79a70-f727-40e1-9dcd-4d959496af42 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.577478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338b721-94c9-41b7-994d-3b76c1c11619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:58.578136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338b721-94c9-41b7-994d-3b76c1c11619 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.578422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338b721-94c9-41b7-994d-3b76c1c11619 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.578710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338b721-94c9-41b7-994d-3b76c1c11619 None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.579862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338b721-94c9-41b7-994d-3b76c1c11619 None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:58.581857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338b721-94c9-41b7-994d-3b76c1c11619 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:58.583917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 63/135] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:56:58 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:58.585075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.585365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.585646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.605344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:56:58.606560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:56:58.676482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:58.678798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a79a70-f727-40e1-9dcd-4d959496af42 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:58.685353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 64/136] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:56:58.686277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.717498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:58.718136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.718423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.718702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/a736dfa50d3841bb971419573615ad37/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.720950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a44c7b4a430429e94e14dfb7e2c81a5, user_id=a736dfa50d3841bb971419573615ad37, role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:58.722567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:58.840647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 73/137] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:56:58 2022] PUT /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/a736dfa50d3841bb971419573615ad37/roles/98057a16b9eb4d96a1931655b2cd15be => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:56:58.841524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc487-eb5f-43ff-b7ce-6748805b8f17 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:58.884717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:58.886254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:58.886735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:58.887387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] PATH_INFO: `/v3/projects/ad00bd6c4e9d494096a9749b38341af2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:58.888983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] RBAC: Authorizing `identity:get_project(project_id=ad00bd6c4e9d494096a9749b38341af2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:58.890739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:58.892751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 65/138] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:56:58 2022] GET /identity/v3/projects/ad00bd6c4e9d494096a9749b38341af2 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:56:58.899408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:58.942645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:59.024943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:59.035034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b1b6-b45e-498e-a6bc-e6c2d74cac60 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:59.074129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:59.074832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:59.075119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:59.075431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] PATH_INFO: `/v3/projects/ad00bd6c4e9d494096a9749b38341af2/users/a736dfa50d3841bb971419573615ad37/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:59.077726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] RBAC: Authorizing `identity:create_grant(project_id=ad00bd6c4e9d494096a9749b38341af2, user_id=a736dfa50d3841bb971419573615ad37, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:59.079528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:59.149752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:59.181899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 74/139] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:56:58 2022] PUT /identity/v3/projects/ad00bd6c4e9d494096a9749b38341af2/users/a736dfa50d3841bb971419573615ad37/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:56:59.208814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e6ce3-1ee6-4a78-a518-869806adeb5a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:59.249774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2f190-5f66-422a-9c34-fb50455868d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:59.250609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2f190-5f66-422a-9c34-fb50455868d7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:59.250991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2f190-5f66-422a-9c34-fb50455868d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:59.251441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2f190-5f66-422a-9c34-fb50455868d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:59.252845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2f190-5f66-422a-9c34-fb50455868d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:59.255038 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2f190-5f66-422a-9c34-fb50455868d7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:59.257058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 66/140] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:56:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:56:59.262730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:56:59.367405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:56:59.401943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5a521-e6e1-4587-8082-ccabf570829e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:56:59.449633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a918df6-229c-4893-b766-5a825712ebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:56:59.450262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a918df6-229c-4893-b766-5a825712ebfc None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:56:59.450542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a918df6-229c-4893-b766-5a825712ebfc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:56:59.450816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a918df6-229c-4893-b766-5a825712ebfc None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:56:59.451586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a918df6-229c-4893-b766-5a825712ebfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:56:59.453032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a918df6-229c-4893-b766-5a825712ebfc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:56:59.610815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 75/141] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:56:59 2022] POST /identity/v3/users => generated 283 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:00.226173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e0db4-9277-48ca-b8ec-5e2d746121cc None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:00.226677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e0db4-9277-48ca-b8ec-5e2d746121cc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:00.226748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e0db4-9277-48ca-b8ec-5e2d746121cc None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:00.228339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 67/142] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:00.242650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:00.242650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:00.242650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:00.267825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:00.269881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:00.353726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 76/143] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:00.665808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e0db4-9277-48ca-b8ec-5e2d746121cc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:00.793446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e0db4-9277-48ca-b8ec-5e2d746121cc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:00.829589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:00.836238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c173e-9ea4-4b69-84ee-0cdbfdf19463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c173e-9ea4-4b69-84ee-0cdbfdf19463 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c173e-9ea4-4b69-84ee-0cdbfdf19463 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c173e-9ea4-4b69-84ee-0cdbfdf19463 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c173e-9ea4-4b69-84ee-0cdbfdf19463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e26c173e-9ea4-4b69-84ee-0cdbfdf19463 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:00.859497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:00.860023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:00.860776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:00.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 68/144] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:00.866274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:00.904116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edac006-b0d2-4502-b6b0-e008875538ea None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:00.904754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edac006-b0d2-4502-b6b0-e008875538ea None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:00.905305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edac006-b0d2-4502-b6b0-e008875538ea None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:00.907538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 69/145] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:00.914227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:00.914811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:00.915374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:00.946730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:00.948472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:00.994364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:01.068835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b55ab0-1f5b-43d9-950d-c2b2f84fc784 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:01.085411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 70/146] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:01.118313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:01.119423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.119907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.120378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.121276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:01.121788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:01.123498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:01.133485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 77/147] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:01.140050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:01.264807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7dc3-7e2b-44dc-af4c-2f6b3c0c0a77 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:01.290876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:01.306711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93cf1-e614-43f1-b04f-df6677067aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:01.307754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93cf1-e614-43f1-b04f-df6677067aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.308242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93cf1-e614-43f1-b04f-df6677067aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.308714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93cf1-e614-43f1-b04f-df6677067aa4 None admin] PATH_INFO: `/v3/users/3502a3cecf6f4e9aac93ab50c72d986b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.311292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93cf1-e614-43f1-b04f-df6677067aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=3502a3cecf6f4e9aac93ab50c72d986b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:01.313048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93cf1-e614-43f1-b04f-df6677067aa4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:01.314875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 71/148] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:01 2022] GET /identity/v3/users/3502a3cecf6f4e9aac93ab50c72d986b => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:01.320777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:01.408057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced4f0-3bd5-4836-b0ed-169ccc160b9a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.408680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced4f0-3bd5-4836-b0ed-169ccc160b9a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.409806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced4f0-3bd5-4836-b0ed-169ccc160b9a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.411392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 72/149] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:01.417072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2662690-0596-4adc-9a30-942b183af72f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.417360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2662690-0596-4adc-9a30-942b183af72f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.417648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2662690-0596-4adc-9a30-942b183af72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.440923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d2662690-0596-4adc-9a30-942b183af72f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:01.443526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d2662690-0596-4adc-9a30-942b183af72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:01.468366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4e464-0010-4b70-98b2-f03a2937db2a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:01.525090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:01.533787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:01.540031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.540031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.540031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.540031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:01.540031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:01.543699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 78/150] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:01 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:01.561041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:01.562740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 73/151] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:01.625116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2662690-0596-4adc-9a30-942b183af72f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:01.752138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874c3a8-e6aa-4fd3-94fb-58da2899da76 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:01.796149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2662690-0596-4adc-9a30-942b183af72f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:01.815147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:01.816086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:01.817490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.817776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.818052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/3502a3cecf6f4e9aac93ab50c72d986b/roles/c099a4b10af549a6ab8cff946dca2e5f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.823934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] RBAC: Authorizing `identity:create_grant(project_id=5a44c7b4a430429e94e14dfb7e2c81a5, user_id=3502a3cecf6f4e9aac93ab50c72d986b, role_id=c099a4b10af549a6ab8cff946dca2e5f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:01.825391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:01.843932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:01.844640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:01.844920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:01.845189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:01.859603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:01.868029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:01.868962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:01.869738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 74/152] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:01.875043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:01.915858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 79/153] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:57:01 2022] PUT /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/users/3502a3cecf6f4e9aac93ab50c72d986b/roles/c099a4b10af549a6ab8cff946dca2e5f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:57:01.995740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ab0bb-8cad-4ee1-9f02-7b61f985b84b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:02.034888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:02.035530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:02.035807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:02.036077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:02.036751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:02.037050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:02.038422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:02.048092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 75/154] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:02.054063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:02.103243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:02.147285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:02.175529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:02.253978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6773-1b92-4698-9671-ea99449e10fa None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:02.282564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6fb22-10c7-4972-be6e-775d56571b34 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:02.348019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:02.348654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:02.348817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:02.348922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:02.349151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:02.349452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:02.349858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:02.350212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:02.350372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:02.351857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:02.354549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 80/155] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:02 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:02.359300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:02.371577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:02.379446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:02.380342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:02.381138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 76/156] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:02.397640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:02.564898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:02.574209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14633d6-337a-4017-8bac-21166715e6dd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:02.590975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa1295-0edf-487a-af35-d31d7303a57e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:02.629534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:02.630642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:02.632154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:02.632625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:02.633127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:02.634738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:02.635988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:02.638298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:02.655659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:02.656614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:02.657176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:02.657641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:02.663502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 77/157] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:02.671032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:02.810680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:02.813667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:02.901164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc17deb-e4ef-4b79-b029-a196fd5856d2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:02.976820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:02.978446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:02.979688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:02.980182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] PATH_INFO: `/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:02.981638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] RBAC: Authorizing `identity:get_user(user_id=47dfc0e354ea47fc9cdf40b73108b8fe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:02.983332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:02.985195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 78/158] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:02 2022] GET /identity/v3/users/47dfc0e354ea47fc9cdf40b73108b8fe => generated 247 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:02.992444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:02.997009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 81/159] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:03.002907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:03.051596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:03.056733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:03.126380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1579d8e-a986-4dff-b2c7-939e73c8ae1e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:03.157466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4b52c-16bb-4726-88b5-b8fde4352a8b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:03.175339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:03.175997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.176280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.176542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.183970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:03.185477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:03.187284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 82/160] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:03 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:03.195857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:03.255865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:03.256487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.256770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.257044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.374504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:03.385976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ff27f-18cb-4823-894d-08ec1e858ea8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:03.408713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:03.410206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:03.455993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:03.457134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.457424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.457698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.458442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=a736dfa50d3841bb971419573615ad37) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:03.458751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:03.460242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483e864-d053-4eaa-aca4-4a8b1a87640b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:03.463499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:03.471287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 83/161] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:03 2022] GET /identity/v3/role_assignments?user.id=a736dfa50d3841bb971419573615ad37&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:03.472782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07262be6-9e1f-468c-b56e-c5fdfec3c467 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.473071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07262be6-9e1f-468c-b56e-c5fdfec3c467 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.473344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07262be6-9e1f-468c-b56e-c5fdfec3c467 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.474726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 84/162] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:03.480648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.480962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.481206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.516283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:03.517517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:03.603965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 79/163] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:03.609500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:03.646839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 85/164] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:03.764893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0302-e209-4035-b83e-7701dcfc694e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:03.803892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:03.809850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d8597-0835-4188-9282-35ec5c9463c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:03.810505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d8597-0835-4188-9282-35ec5c9463c1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.810785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d8597-0835-4188-9282-35ec5c9463c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.811057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d8597-0835-4188-9282-35ec5c9463c1 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.816770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d8597-0835-4188-9282-35ec5c9463c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:03.818217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d8597-0835-4188-9282-35ec5c9463c1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:03.820024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 80/165] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:03 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:03.826991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:03.896780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09046d-9e2e-4742-ac95-ca0c74bcd8ee None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.897076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09046d-9e2e-4742-ac95-ca0c74bcd8ee None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.897338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09046d-9e2e-4742-ac95-ca0c74bcd8ee None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.898728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 81/166] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:03.904672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:03.904956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:03.905228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:03.939558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:03.941353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:03.966003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:03.986886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7bdfc-c23e-4375-ab60-9c2c2432a9e3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:04.038872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:04.039475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:04.039754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:04.040027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:04.041014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=47dfc0e354ea47fc9cdf40b73108b8fe) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:04.041321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:04.043526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:04.056737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 86/167] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:03 2022] GET /identity/v3/role_assignments?user.id=47dfc0e354ea47fc9cdf40b73108b8fe&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:04.083436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 82/168] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:04.213357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:04.344456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5b9cd-f372-44df-b62f-988018af8bd5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:04.389874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:04.390356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:04.390356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:04.390621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:04.411921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:04.417713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:04.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:04.419546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 87/169] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:04.424955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:04.610895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b080c-afdc-47d0-a0db-693024f16326 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:04.644598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:04.653106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:04.665297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:04.665297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:04.665297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:04.665297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:04.665297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:04.665912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:04.665991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:04.669466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 83/170] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:04.683101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:04.690194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:04.910637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12566cf5-13f6-4fac-b5ae-6312b7630043 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:04.920852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408049a7-4e9e-49ee-bee9-bf37d077ba43 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.001317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.001970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.002250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.002523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.003748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:05.004297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff552e9a-1803-45b2-add5-90e4e2741556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.005193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:05.005419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff552e9a-1803-45b2-add5-90e4e2741556 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.006131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff552e9a-1803-45b2-add5-90e4e2741556 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.006845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff552e9a-1803-45b2-add5-90e4e2741556 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.007099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 84/171] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:04 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:05.016139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:05.036806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff552e9a-1803-45b2-add5-90e4e2741556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-ff552e9a-1803-45b2-add5-90e4e2741556 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:05.042466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:05.043418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:05.044213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 88/172] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:05.052330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bc0d4-c601-49c2-84ea-e0e97c70d4c9 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.052617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bc0d4-c601-49c2-84ea-e0e97c70d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.052893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bc0d4-c601-49c2-84ea-e0e97c70d4c9 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.054208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 89/173] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:05.058737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bc0d4-c601-49c2-84ea-e0e97c70d4c9 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:05.170158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:05.171939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:05.235784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bc0d4-c601-49c2-84ea-e0e97c70d4c9 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.248248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9316039-80f5-4488-83e8-e36ddc4f775b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.280603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.281329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.281607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.281876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.282540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:05.282842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:05.285182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07358b9e-c996-4420-b6a0-faed81c3a474 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:05.293990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb90002-988d-4cf6-af0e-fe923e34df8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.294568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 90/174] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:05.295202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90002-988d-4cf6-af0e-fe923e34df8b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.295703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90002-988d-4cf6-af0e-fe923e34df8b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.295836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812fe8c-056a-40bc-bfce-5d85e5403a9c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.296213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90002-988d-4cf6-af0e-fe923e34df8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.296433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812fe8c-056a-40bc-bfce-5d85e5403a9c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.296579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812fe8c-056a-40bc-bfce-5d85e5403a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.316077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4812fe8c-056a-40bc-bfce-5d85e5403a9c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:05.317619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4812fe8c-056a-40bc-bfce-5d85e5403a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:05.385224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb90002-988d-4cf6-af0e-fe923e34df8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:05.388172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb90002-988d-4cf6-af0e-fe923e34df8b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:05.411700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 91/175] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:05.413968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4651b1-cf92-4400-81ef-31ef79a8c8d3 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.414395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4651b1-cf92-4400-81ef-31ef79a8c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.414815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4651b1-cf92-4400-81ef-31ef79a8c8d3 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.416532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 92/176] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:05.421356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4651b1-cf92-4400-81ef-31ef79a8c8d3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:05.459333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:05.493603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:05.525786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 85/177] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:05.527590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.528109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.528789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.551478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:05.554054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:05.566481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4651b1-cf92-4400-81ef-31ef79a8c8d3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.637002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 86/178] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:05.638294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:05.641282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.641930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.643101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.643101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] PATH_INFO: `/v3/users/a736dfa50d3841bb971419573615ad37` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.646103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] RBAC: Authorizing `identity:get_user(user_id=a736dfa50d3841bb971419573615ad37)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:05.647600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:05.649189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 93/179] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/users/a736dfa50d3841bb971419573615ad37 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:05.660061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:05.755412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:05.771298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea5675-99d9-4c0f-93b5-81f0724c7ff1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.814949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.815921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.816362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.816797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.818091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:05.819890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:05.822278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 87/180] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:05.823227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:05.831506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e098-1ad5-4612-96eb-942d6ce6a612 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.863950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:05.868908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f1462-6ec7-4298-a9b6-8f5e36a8572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.869518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f1462-6ec7-4298-a9b6-8f5e36a8572e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.870021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f1462-6ec7-4298-a9b6-8f5e36a8572e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.870438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f1462-6ec7-4298-a9b6-8f5e36a8572e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.934915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce12e98-de3d-45c2-a088-08f4d92de3e4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:05.948818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f1462-6ec7-4298-a9b6-8f5e36a8572e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:05.950291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f1462-6ec7-4298-a9b6-8f5e36a8572e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:05.971099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421a51d-9cf9-4d25-bf19-e9cc67e2b17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:05.972442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421a51d-9cf9-4d25-bf19-e9cc67e2b17a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.972725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421a51d-9cf9-4d25-bf19-e9cc67e2b17a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.972999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421a51d-9cf9-4d25-bf19-e9cc67e2b17a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.983764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421a51d-9cf9-4d25-bf19-e9cc67e2b17a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1421a51d-9cf9-4d25-bf19-e9cc67e2b17a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:05.989093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:05.990142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:05.991107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 88/181] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:05.991952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df8074-9c6b-43ed-bb12-774987ce55d2 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:05.991952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df8074-9c6b-43ed-bb12-774987ce55d2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:05.991952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df8074-9c6b-43ed-bb12-774987ce55d2 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:05.995217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 89/182] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:05.995217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df8074-9c6b-43ed-bb12-774987ce55d2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.013105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.055254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 94/183] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:06.056442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb757b-4e7d-4c21-a0c6-1c5ce9b95167 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.056798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb757b-4e7d-4c21-a0c6-1c5ce9b95167 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.057080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb757b-4e7d-4c21-a0c6-1c5ce9b95167 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.058554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 95/184] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:06.060521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb757b-4e7d-4c21-a0c6-1c5ce9b95167 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.094177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.102271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df8074-9c6b-43ed-bb12-774987ce55d2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.136769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.137373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.137651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.137921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.138634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] RBAC: Adding query filter params (role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=a736dfa50d3841bb971419573615ad37) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:06.138958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.140414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.146564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 90/185] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:05 2022] GET /identity/v3/role_assignments?user.id=a736dfa50d3841bb971419573615ad37&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.147450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.167241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb757b-4e7d-4c21-a0c6-1c5ce9b95167 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.205364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1f3dc-19b6-4347-b891-24f65d719540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.206744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1f3dc-19b6-4347-b891-24f65d719540 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.207023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1f3dc-19b6-4347-b891-24f65d719540 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.207328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1f3dc-19b6-4347-b891-24f65d719540 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.220029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1f3dc-19b6-4347-b891-24f65d719540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-16c1f3dc-19b6-4347-b891-24f65d719540 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:06.225218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:06.226151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:06.226937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 96/186] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:06.228244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.228244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.228363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.248469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:06.249856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:06.257624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.278992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe6344-4370-4a69-92c0-bc5fc30409ef None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.320315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.320935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.321218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.321487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.322151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:06.322454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.323874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216ad2b-5763-4d4b-9e3d-d3eb9d378e97 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.323980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 97/187] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:06.324777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.333136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 91/188] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.334294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.334586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.334862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.353989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:06.355045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:06.356920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.458435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5461-e9b1-4250-95a8-5aa1059f7c73 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.459799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 92/189] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:06.464011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.508348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.508995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.509274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.509545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] PATH_INFO: `/v3/projects/ad00bd6c4e9d494096a9749b38341af2` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.511221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] RBAC: Authorizing `identity:get_project(project_id=ad00bd6c4e9d494096a9749b38341af2)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.513235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.515675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 98/190] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/projects/ad00bd6c4e9d494096a9749b38341af2 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.520036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.558018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.598079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a1f3e-d9a9-4d04-91c9-82205ad99988 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.617978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.628694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d633b6-f4d7-46e3-81d8-8700b0d472ef None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.636274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.636895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.637179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.637457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.638123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:06.638428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.639910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.648360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 93/191] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.653154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.668358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.669006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.669308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.669585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] PATH_INFO: `/v3/users/285d36b03f644df0a6d5867876ff4af8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.683803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=285d36b03f644df0a6d5867876ff4af8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.685245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.686860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 99/192] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/users/285d36b03f644df0a6d5867876ff4af8 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.692684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.761191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e976f4a-d14b-4c68-a50b-8c717e9f5dbe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.793550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.794151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.794425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.794692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.795435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=ad00bd6c4e9d494096a9749b38341af2, user.id=a736dfa50d3841bb971419573615ad37) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:06.795742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.796306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfa6ce-63ae-45a7-bab2-6e1b2c7a5fe3 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.797174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.798873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.803391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 94/193] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/role_assignments?user.id=a736dfa50d3841bb971419573615ad37&scope.project.id=ad00bd6c4e9d494096a9749b38341af2&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.804158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.839975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.840579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.840857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.841125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.842297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.843836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.845510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 100/194] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.846308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.867258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:06.919529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e34cd6-e918-4c10-aad3-10a657a3a547 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:06.962465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:06.963120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:06.963437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:06.963715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] PATH_INFO: `/v3/users/285d36b03f644df0a6d5867876ff4af8` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:06.964910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] RBAC: Authorizing `identity:get_user(user_id=285d36b03f644df0a6d5867876ff4af8)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:06.966393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:06.968208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 95/195] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/users/285d36b03f644df0a6d5867876ff4af8 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:06.968913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:06.972081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad98b7-ce45-4537-8abb-bed423b37445 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.011584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.012209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.012492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.012766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.047239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.081894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1891e7-dc83-48f1-b850-afbd2d40c110 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.085639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.087153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.104532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.116266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.116901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.118466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.118466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.127314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:07.133161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:07.134083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:07.134859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 96/196] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:07.135590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.188133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 101/197] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:07.189094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.235979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf3448-487c-420a-bd5e-9624dac46e96 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.261210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.270651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.273844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.273844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.273844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.273844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.274347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.293994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159475-7dc4-4b99-9f35-122f3ddee523 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.311358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.328166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.328766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.329046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.329337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.353876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 97/198] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:07 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:07.354847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.428127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.430216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.473590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ff89-d2b6-4c1d-a708-2d436a716600 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.513761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.514380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.514654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.514925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.515623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:07.515925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.517318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.528440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 98/199] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:07.529272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.542900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 102/200] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:07.543623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.566819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.579796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.649024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3818063-d3c6-4a9e-98f9-3ab520aff44d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.676859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce00db-95bf-4258-adaa-55cf4e1cf466 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.692780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.693576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.694271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.694555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.695836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.697357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.699116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 99/201] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:07.699751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.722219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.722874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.723172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.723455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] PATH_INFO: `/v3/users/3502a3cecf6f4e9aac93ab50c72d986b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.724609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] RBAC: Authorizing `identity:get_user(user_id=3502a3cecf6f4e9aac93ab50c72d986b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.726049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.728060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 103/202] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/users/3502a3cecf6f4e9aac93ab50c72d986b => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:07.728912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.821761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f1634-4da1-4d58-9315-bef35e70640c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.825200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.836097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:07.843261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d253c8-d1f4-493f-b835-4e8cc0a373bc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:07.859281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.859977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.860255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.860583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.862735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.864235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.866539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 100/203] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:07.872128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.884092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:07.885037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:07.885552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:07.886019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:07.886941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=285d36b03f644df0a6d5867876ff4af8) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:07.887509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:07.889153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:07.895250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 104/204] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/role_assignments?user.id=285d36b03f644df0a6d5867876ff4af8&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:07.897494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:07.987041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed768b-4da2-4a75-ab4b-3bd39939b86f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:08.040426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af72a-7485-4dd3-90b2-4ed0ffb3362a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:08.065025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:08.065879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:08.065988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:08.066233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:08.088327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:08.089339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:08.089879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:08.090408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:08.091514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] RBAC: Adding query filter params (role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=285d36b03f644df0a6d5867876ff4af8) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:08.092082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:08.093780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:08.103203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 105/205] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/role_assignments?user.id=285d36b03f644df0a6d5867876ff4af8&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:08.155811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:08.180183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:08.181698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:08.308815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 101/206] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:08.315333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:08.333222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:08.446636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2b81f-1e61-4e76-bff6-90f36bb62745 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:08.485962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3743d-f83e-4df5-8d93-eae44ee4e361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:08.486607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3743d-f83e-4df5-8d93-eae44ee4e361 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:08.486885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3743d-f83e-4df5-8d93-eae44ee4e361 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:08.487209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3743d-f83e-4df5-8d93-eae44ee4e361 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:08.488343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3743d-f83e-4df5-8d93-eae44ee4e361 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:08.489799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3743d-f83e-4df5-8d93-eae44ee4e361 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:08.491449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 106/207] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:08 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:08.500259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:08.623039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:08.647466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0cebf-aaa6-4ea1-bf13-ff2e94c3ebc5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:08.697821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:08.698441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:08.698734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:08.699011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:08.699775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] RBAC: Adding query filter params (role.id=c099a4b10af549a6ab8cff946dca2e5f, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5, user.id=3502a3cecf6f4e9aac93ab50c72d986b) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:08.700089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:08.701523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66227f4-ecfc-4c98-8074-4930cefa1d47 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:08.711892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 102/208] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:08 2022] GET /identity/v3/role_assignments?user.id=3502a3cecf6f4e9aac93ab50c72d986b&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=c099a4b10af549a6ab8cff946dca2e5f => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:09.891038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616a552-06f8-4464-9beb-6494b7f34a7c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:09.891581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334e2d0-94ed-4bba-831a-8b2f9948b2de None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:09.891581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334e2d0-94ed-4bba-831a-8b2f9948b2de None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:09.892012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334e2d0-94ed-4bba-831a-8b2f9948b2de None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:09.892633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616a552-06f8-4464-9beb-6494b7f34a7c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:09.893631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616a552-06f8-4464-9beb-6494b7f34a7c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:09.893731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 103/209] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:09.895347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 107/210] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:09.898643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:09.898928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:09.899236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:09.903032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:09.903348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:09.903626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:09.920878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:09.922474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:09.923332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:09.924550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:09.997765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 108/211] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:10.004489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 104/212] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:10.274222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.276931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.286881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:10.372202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:10.393886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5f66c-36e9-4ec7-83ab-b46c3d6db78f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:10.410838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874510a1-0245-49ac-af2d-5eb3bd2508db None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:10.432534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc107eb-dec4-43b9-9111-c598ff3282f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:10.433182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc107eb-dec4-43b9-9111-c598ff3282f7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.433465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc107eb-dec4-43b9-9111-c598ff3282f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.433737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc107eb-dec4-43b9-9111-c598ff3282f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.445322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc107eb-dec4-43b9-9111-c598ff3282f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:10.449239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872f86b-7a16-418b-92ad-8bcc52ecc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:10.449878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872f86b-7a16-418b-92ad-8bcc52ecc9bf None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.450158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872f86b-7a16-418b-92ad-8bcc52ecc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-dcc107eb-dec4-43b9-9111-c598ff3282f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:10.450252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:10.451061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:10.451935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:10.452531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872f86b-7a16-418b-92ad-8bcc52ecc9bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.452597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 109/213] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:10.452597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448c223-c2f4-4a46-b4e0-adf68264d5ba None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.453070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448c223-c2f4-4a46-b4e0-adf68264d5ba None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.453070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448c223-c2f4-4a46-b4e0-adf68264d5ba None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.454519 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 110/214] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:10.456412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872f86b-7a16-418b-92ad-8bcc52ecc9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:10.457802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448c223-c2f4-4a46-b4e0-adf68264d5ba None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.458662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872f86b-7a16-418b-92ad-8bcc52ecc9bf None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:10.460561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 105/215] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:10.463351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.463858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.464333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.484440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:10.485937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:10.538067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:10.567362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 106/216] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:10.567362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.574955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448c223-c2f4-4a46-b4e0-adf68264d5ba None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:10.615257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:10.616426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.616849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.617510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.618808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:10.619561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:10.621360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85406647-a610-402e-92bf-ebdaa3f8db7b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:10.636701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 111/217] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:10.636701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596303a-6138-42e4-9bea-7acff6a30b4d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.636701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596303a-6138-42e4-9bea-7acff6a30b4d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.636701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596303a-6138-42e4-9bea-7acff6a30b4d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.636701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 112/218] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:10.638091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596303a-6138-42e4-9bea-7acff6a30b4d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.652184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:10.687930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1eca8a-dc96-4af7-942d-bc7fef9bd8c8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:10.723175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:10.723894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.724250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.724597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.725316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:10.726821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:10.748278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596303a-6138-42e4-9bea-7acff6a30b4d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:10.777315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:10.781972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a41882-8c2c-4acf-9e0a-fc398a0c0aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:10.782602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41882-8c2c-4acf-9e0a-fc398a0c0aea None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.782887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41882-8c2c-4acf-9e0a-fc398a0c0aea None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.783178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41882-8c2c-4acf-9e0a-fc398a0c0aea None admin] PATH_INFO: `/v3/users/93fe86e531d04596bc7e1b101376a212` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.796509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a41882-8c2c-4acf-9e0a-fc398a0c0aea None admin] RBAC: Authorizing `identity:get_user(user_id=93fe86e531d04596bc7e1b101376a212)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:10.797781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a41882-8c2c-4acf-9e0a-fc398a0c0aea None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:10.799463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 113/219] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:10 2022] GET /identity/v3/users/93fe86e531d04596bc7e1b101376a212 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:10.800659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.800950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.801225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:10.819041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:10.820261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:10.821185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 107/220] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:10 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:10.823738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.912293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 114/221] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:10.914021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:10.952775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee029d3-c0c8-4987-84ba-83e3be16836a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:10.986605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:10.996056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:10.996671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:10.996951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:10.997217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.049890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff1de-8a05-45ef-bc10-aa447a72184e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.076034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:11.086439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.088172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.092857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.093503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.093783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.094059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.105884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:11.111287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:11.113392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:11.114387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:11.118882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 115/222] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:11.119890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.248948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f21e1-b52b-4c4e-9b33-44158c624f3d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.286079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 108/223] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:11.286877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.288677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.289301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.289590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.289884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.290912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:11.291250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.292935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.303837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 116/224] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:11.304597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.330659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:11.353637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:11.421375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4232f-b2f6-4588-bccb-0942325087c4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.422590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c337251-1886-4818-ab7a-ab4e37559890 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.468207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.469305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.469689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.470136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.472047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.473476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.473600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.474244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.474594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.474938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.475272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 117/225] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:11.476428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.489729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:11.496914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:11.497780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:11.498554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:11.499143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 109/226] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:11.500631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.606736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:11.612429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95064d6-0a6e-4ed2-9814-e0c22b6219cb None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.620944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3dfd6-b2f7-48a0-8229-4edea4efb05c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.626529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:11.656938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.657585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.657871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.658147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] PATH_INFO: `/v3/users/285d36b03f644df0a6d5867876ff4af8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.658536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.659860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] RBAC: Authorizing `identity:get_user(user_id=285d36b03f644df0a6d5867876ff4af8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.660117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.660599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.661059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.661619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.662481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=93fe86e531d04596bc7e1b101376a212) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:11.662986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.663622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 118/227] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/users/285d36b03f644df0a6d5867876ff4af8 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:11.664399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.665334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.671826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 110/228] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/role_assignments?user.id=93fe86e531d04596bc7e1b101376a212&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:11.680416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.789963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53ad77-596c-400a-9a91-970c820f7872 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.832735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.833374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.833661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.833939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.834787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:11.835106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.836653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.848575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 119/229] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:11.856256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:11.901778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:11.937357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99829f41-9113-4131-9008-5d980d74aca6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.981456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4655f1-9078-45b1-95a4-404357f68e76 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:11.989566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:11.990775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:11.991371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:11.991787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:11.993780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:11.995683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:11.998802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 111/230] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:12.006450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:12.040121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:12.049402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:12.050114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:12.050398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:12.052812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] PATH_INFO: `/v3/users/285d36b03f644df0a6d5867876ff4af8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:12.055939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] RBAC: Authorizing `identity:get_user(user_id=285d36b03f644df0a6d5867876ff4af8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:12.057343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:12.059312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 120/231] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:11 2022] GET /identity/v3/users/285d36b03f644df0a6d5867876ff4af8 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:12.064761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:12.138781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181dea48-0357-495c-a483-c244a4b7949f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:12.182277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9c82b-ed97-42ff-ad73-c59888ab998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:12.182912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9c82b-ed97-42ff-ad73-c59888ab998c None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:12.183209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9c82b-ed97-42ff-ad73-c59888ab998c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:12.183486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9c82b-ed97-42ff-ad73-c59888ab998c None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/285d36b03f644df0a6d5867876ff4af8/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:12.185408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9c82b-ed97-42ff-ad73-c59888ab998c None admin] RBAC: Authorizing `identity:create_grant(project_id=4f84c0c22d344bdba56d9d3dccd426d0, user_id=285d36b03f644df0a6d5867876ff4af8, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:12.186849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9c82b-ed97-42ff-ad73-c59888ab998c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:12.210389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3551-5ea3-4703-ad70-38d62ed957a8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:12.220119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:12.246287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:12.246923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:12.247229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:12.247506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:12.248657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:12.250094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:12.252092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 121/232] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:12 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:12.257705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:12.269308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 112/233] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:57:12 2022] PUT /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/285d36b03f644df0a6d5867876ff4af8/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:12.386426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceff6e5-a555-4740-808c-56cb47773509 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:12.421207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc469b15-0406-473b-9583-1398f3845b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:12.421946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469b15-0406-473b-9583-1398f3845b37 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:12.422231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469b15-0406-473b-9583-1398f3845b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:12.422559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469b15-0406-473b-9583-1398f3845b37 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/285d36b03f644df0a6d5867876ff4af8/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:12.425429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc469b15-0406-473b-9583-1398f3845b37 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f84c0c22d344bdba56d9d3dccd426d0, user_id=285d36b03f644df0a6d5867876ff4af8, role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:12.426923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc469b15-0406-473b-9583-1398f3845b37 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:12.530098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 122/234] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:57:12 2022] PUT /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/285d36b03f644df0a6d5867876ff4af8/roles/98057a16b9eb4d96a1931655b2cd15be => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:13.503596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d09f-73fa-4f83-9899-868a6e5ffcfe None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:13.504097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d09f-73fa-4f83-9899-868a6e5ffcfe None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:13.504212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d09f-73fa-4f83-9899-868a6e5ffcfe None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:13.505781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 113/235] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:13.511186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e58dd-89a2-4cc7-bcf3-355df71f8dc0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:13.511591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e58dd-89a2-4cc7-bcf3-355df71f8dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:13.511757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e58dd-89a2-4cc7-bcf3-355df71f8dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:13.540411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-640e58dd-89a2-4cc7-bcf3-355df71f8dc0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:13.541916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-640e58dd-89a2-4cc7-bcf3-355df71f8dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:13.577535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:13.628743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 123/236] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:13.681857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240c368-c383-4d4e-947b-0fc962c0b574 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:13.682135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240c368-c383-4d4e-947b-0fc962c0b574 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:13.682311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240c368-c383-4d4e-947b-0fc962c0b574 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:13.683709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 114/237] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:13.689179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:13.689480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:13.690301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:13.720133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:13.721400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:13.815426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 124/238] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:13.939699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240c368-c383-4d4e-947b-0fc962c0b574 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:14.047602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:14.069013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240c368-c383-4d4e-947b-0fc962c0b574 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:14.104725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:14.105789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.106279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.106747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.108135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:14.109852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:14.112212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 115/239] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:14.115258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:14.121086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:14.242333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3e6c0-6ea6-4e7b-afe0-4bef70453365 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:14.284232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57bd2-d867-4148-b976-c013436e0e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:14.285185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57bd2-d867-4148-b976-c013436e0e5d None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.285678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57bd2-d867-4148-b976-c013436e0e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.286157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57bd2-d867-4148-b976-c013436e0e5d None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.289871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c57bd2-d867-4148-b976-c013436e0e5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:14.291859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c57bd2-d867-4148-b976-c013436e0e5d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:14.310337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:14.323196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd427bd-beae-4c46-856b-c3abea2269f1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:14.354554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:14.355215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.355552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.355825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.367046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:14.371987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:14.372862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:14.373635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:14.378612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 125/240] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:14.379851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:14.427987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 116/241] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:14 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:14.429958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bff8c-a96f-442a-b787-d88f99085673 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.430320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bff8c-a96f-442a-b787-d88f99085673 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.430665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bff8c-a96f-442a-b787-d88f99085673 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.439096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 117/242] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:14.445191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.445554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.445924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.460817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:14.481536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:14.482879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:14.506273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b352a8a-406d-4471-965f-abffb08f7dfa None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:14.566765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:14.568197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.568650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.569131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.569977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:14.570448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:14.572246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:14.594504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 126/243] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:14.598363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:14.637939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 118/244] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:14.639679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:14.724490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a813a30-87c4-4465-b4ed-dc9b0e201d48 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:14.762650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:14.763609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.764063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.764503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] PATH_INFO: `/v3/users/4103adfeddda43a9b50ac1ce90de1cdc` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.778575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] RBAC: Authorizing `identity:get_user(user_id=4103adfeddda43a9b50ac1ce90de1cdc)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:14.780337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:14.782222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 127/245] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:14 2022] GET /identity/v3/users/4103adfeddda43a9b50ac1ce90de1cdc => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:14.788722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:14.903366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460960f-6d80-456a-9b06-b315f4d25a12 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:14.942417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9bde3-11e2-45c7-b450-8eb4b8361473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:14.943383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9bde3-11e2-45c7-b450-8eb4b8361473 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:14.943832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9bde3-11e2-45c7-b450-8eb4b8361473 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:14.944273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9bde3-11e2-45c7-b450-8eb4b8361473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:14.947208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:14.972945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.026115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9bde3-11e2-45c7-b450-8eb4b8361473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:15.027644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9bde3-11e2-45c7-b450-8eb4b8361473 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:15.101971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:15.138556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fcb1c-f7e2-47b2-97db-bf276e3a048d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:15.187749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:15.188857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.189146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.190420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.199379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 119/246] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:15.201009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e0943-19b4-432a-9e66-c3c4dbcccc3b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.201231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e0943-19b4-432a-9e66-c3c4dbcccc3b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.201512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e0943-19b4-432a-9e66-c3c4dbcccc3b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.202892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 120/247] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:15.210396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.210713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.210947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.211900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:15.217138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:15.218079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:15.218860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 128/248] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:15.219632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.234964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:15.236265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:15.267418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:15.311657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 121/249] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:15.312457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.356407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5e131-a63f-43c9-8c75-ec63e41796da None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:15.431199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c514862-10f8-40b6-885f-cf3ee8ee360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:15.431915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c514862-10f8-40b6-885f-cf3ee8ee360d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.432198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c514862-10f8-40b6-885f-cf3ee8ee360d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.432469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c514862-10f8-40b6-885f-cf3ee8ee360d None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.434084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c514862-10f8-40b6-885f-cf3ee8ee360d None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:15.435561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c514862-10f8-40b6-885f-cf3ee8ee360d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:15.438078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 129/250] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:15 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:15.440618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95970b-40f5-4e4d-b4b9-755b839936e7 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.440899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95970b-40f5-4e4d-b4b9-755b839936e7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.441173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95970b-40f5-4e4d-b4b9-755b839936e7 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.443888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 130/251] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:15.444611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95970b-40f5-4e4d-b4b9-755b839936e7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.463969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:15.501162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964a5c-6d35-44bb-b3ef-9d804b56de66 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:15.563250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:15.563799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.564076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.564344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.565498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:15.565798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:15.567178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408ed07-9b3b-4380-86b0-52fa48259331 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:15.584430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 122/252] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:15.585677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.585966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.586243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.624069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95970b-40f5-4e4d-b4b9-755b839936e7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:15.624931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:15.626570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:15.654018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:15.666733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:15.667699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.668179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.668648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.669569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] RBAC: Adding query filter params (role.id=c099a4b10af549a6ab8cff946dca2e5f, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=4103adfeddda43a9b50ac1ce90de1cdc) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:15.670111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:15.671791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:15.678260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 131/253] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:15 2022] GET /identity/v3/role_assignments?user.id=4103adfeddda43a9b50ac1ce90de1cdc&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=c099a4b10af549a6ab8cff946dca2e5f => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:15.683766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.710928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 123/254] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:15.711804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.826235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80996e35-e4e1-4b92-a652-2b73c1756d0e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:15.860975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:15.865438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:15.866305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:15.866585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:15.866858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:15.883188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:15.884239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:15.885093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:15.885845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 124/255] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:15.892718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:15.935203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:15.973499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8170b49-7a0d-4604-b578-513c0781afaf None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.008877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fdd69-6770-45b0-84cf-fe561c931528 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.048112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.048993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.051425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.051815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.055187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:16.055187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:16.055187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:16.065458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.066151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 125/256] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:16.066655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.067002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.067382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.067846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] PATH_INFO: `/v3/users/93fe86e531d04596bc7e1b101376a212` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.074015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] RBAC: Authorizing `identity:get_user(user_id=93fe86e531d04596bc7e1b101376a212)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:16.075874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:16.078827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 132/257] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:15 2022] GET /identity/v3/users/93fe86e531d04596bc7e1b101376a212 => generated 297 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:16.084655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.218982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:16.257916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b58370-12ba-4f33-b2f1-aa71ddac90fe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.296682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.298362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.299022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.301225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.321197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:16.329471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:16.330846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:16.331717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:16.334502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 126/258] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:16.336841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab52c-9693-4530-bb6b-3d1839b157e9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.339233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.427919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.428586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.428873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.429146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] PATH_INFO: `/v3/users/285d36b03f644df0a6d5867876ff4af8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.430601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] RBAC: Authorizing `identity:get_user(user_id=285d36b03f644df0a6d5867876ff4af8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:16.432084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:16.439211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 133/259] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:16 2022] GET /identity/v3/users/285d36b03f644df0a6d5867876ff4af8 => generated 283 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:16.439211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.451204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:16.460877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b386-bc8a-4498-a338-b7fe9113ed64 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.497625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.498611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.499115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.499631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.500964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:16.502693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:16.504677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 127/260] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:16 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:16.505663 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.515129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:16.610545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312931d9-1238-4ccf-b6b9-58d8385b1c34 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.646850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.647835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.648340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.648803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.721351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6175010-ae57-4602-be47-bf21ef917353 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.722561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:16.724569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:16.734319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:16.824474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:16.825079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:16.825372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:16.825654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:16.826357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:16.826646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:16.828067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:16.833462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 128/261] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:16.834504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.846340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 134/262] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:16.847152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:16.942025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6138b-dc30-44d8-8871-0cdfb686fe04 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:16.943877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:16.959197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:16.983211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aeb01-2bd2-46f1-9ebc-9c313ee4ad6c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:17.014512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:17.015151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.015539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.015722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.016856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:17.018924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:17.020601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 135/263] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:16 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:17.021452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:17.039883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:17.040835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.041127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.041412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/93fe86e531d04596bc7e1b101376a212/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.047938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f84c0c22d344bdba56d9d3dccd426d0, user_id=93fe86e531d04596bc7e1b101376a212, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:17.049445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:17.125290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4926c6-9c91-41c7-bb9f-60c6e2b9a739 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:17.160599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:17.161241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.161348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 129/264] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:57:16 2022] PUT /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/93fe86e531d04596bc7e1b101376a212/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:17.162196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:17.165295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.166062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] PATH_INFO: `/v3/users/285d36b03f644df0a6d5867876ff4af8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.169767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] RBAC: Authorizing `identity:get_user(user_id=285d36b03f644df0a6d5867876ff4af8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:17.171274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:17.173310 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 136/265] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:17 2022] GET /identity/v3/users/285d36b03f644df0a6d5867876ff4af8 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:17.179495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:17.205956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:17.357317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0054fe7-45b2-447c-b46d-7bc6d117c958 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:17.366468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890905fe-e105-4c8b-a84d-18023f9d98c2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:17.429469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:17.430098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.430482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.430754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.441716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:17.442473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.442760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.443032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.443847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=285d36b03f644df0a6d5867876ff4af8) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:17.444158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:17.445702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571dce2-57b3-499a-937b-d4e1bd54f727 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:17.458925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 130/266] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:17 2022] GET /identity/v3/role_assignments?user.id=285d36b03f644df0a6d5867876ff4af8&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:17.544385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:17.545884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:17.548096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:17.682268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f29caa-a2c3-4aa7-9634-e7fa2f79fba4 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.682616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f29caa-a2c3-4aa7-9634-e7fa2f79fba4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.683000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f29caa-a2c3-4aa7-9634-e7fa2f79fba4 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.684661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 131/267] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:17.689240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 137/268] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:17.689569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b936d1-66b9-4720-bcdd-b89c7c38a964 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.689926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b936d1-66b9-4720-bcdd-b89c7c38a964 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.690213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b936d1-66b9-4720-bcdd-b89c7c38a964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.694529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:17.718218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-68b936d1-66b9-4720-bcdd-b89c7c38a964 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:17.719408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-68b936d1-66b9-4720-bcdd-b89c7c38a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:17.787546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:17.869661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 132/269] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:17.871408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b863f9-ca11-471b-86ab-1daca3b9ce6b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.871904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b863f9-ca11-471b-86ab-1daca3b9ce6b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.872380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b863f9-ca11-471b-86ab-1daca3b9ce6b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.873917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 133/270] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:17.879674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.880244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.880720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.882038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31db4db-12cb-4417-a639-f425eb9f757f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:17.884477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:17.910742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:17.911986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:17.953780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:17.954435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:17.954716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:17.954990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:17.959919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:17.961459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:17.964084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 138/271] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:17 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:17.970103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:18.041253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 134/272] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:18.105796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be16787-6620-480e-b5e0-ea037b10f22d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:18.145054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:18.145673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.145956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.146228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.146959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] RBAC: Adding query filter params (role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=285d36b03f644df0a6d5867876ff4af8) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:18.147298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:18.148749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:18.156999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 139/273] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:17 2022] GET /identity/v3/role_assignments?user.id=285d36b03f644df0a6d5867876ff4af8&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:18.184471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:18.298346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:18.312808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e3a66-06aa-4917-8c8e-63bb34462499 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:18.352678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:18.353335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.353618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.353893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.355124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:18.356646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:18.358677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 135/274] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:18.364129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:18.412121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:18.428041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:18.507717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a873cb-7ec4-4446-b2b9-1d05a613681c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:18.538293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5fb88-07c5-4c7a-af80-5361be6936c0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:18.548241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9d1bc-0ac5-4db1-9b47-9592cc3da1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:18.549048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d1bc-0ac5-4db1-9b47-9592cc3da1bc None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.549293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d1bc-0ac5-4db1-9b47-9592cc3da1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.549575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d1bc-0ac5-4db1-9b47-9592cc3da1bc None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.550425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9d1bc-0ac5-4db1-9b47-9592cc3da1bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:18.552049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9d1bc-0ac5-4db1-9b47-9592cc3da1bc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:18.576874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:18.577540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.577825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.578099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.595391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:18.601606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:18.602571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:18.603480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 136/275] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:18.609547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:18.631497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:18.663025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 140/276] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:18 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:18.710936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c039a-d8ba-4cf3-af76-3efb9b8d6af6 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.711485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c039a-d8ba-4cf3-af76-3efb9b8d6af6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.711769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c039a-d8ba-4cf3-af76-3efb9b8d6af6 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.713371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 141/277] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:18.719453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.720081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.720553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.742678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:18.745014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:18.753097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24230b-6f01-4aa0-9fba-c3d4d8ffc842 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:18.802520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:18.803961 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.804506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.804975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.805870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:18.806992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:18.808725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:18.820822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 137/278] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:18.826512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:18.847926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 142/279] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:18.914407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:18.934335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2d6e2-54d8-4fac-a46a-97a32fe8394f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:18.970480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:18.971398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:18.971847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:18.972281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] PATH_INFO: `/v3/users/60116d6927ec483fa137342e950110c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:18.986151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] RBAC: Authorizing `identity:get_user(user_id=60116d6927ec483fa137342e950110c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:18.987854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:18.990043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 138/280] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:18 2022] GET /identity/v3/users/60116d6927ec483fa137342e950110c1 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:18.995531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:19.025117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:19.120722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95767ad0-a9d0-43e3-9c5d-1ef575ea9ea4 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:19.125382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:19.167807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:19.168427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:19.169176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:19.169176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:19.169690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, user.id=60116d6927ec483fa137342e950110c1, scope.system=all) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:19.169999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:19.171464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:19.177182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 143/281] 10.222.0.62 () {56 vars in 1308 bytes} [Tue Sep 13 09:57:18 2022] GET /identity/v3/role_assignments?user.id=60116d6927ec483fa137342e950110c1&scope.system=all&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:19.282297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27ed5e-c92a-492b-ad49-e5a714ee0673 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:19.329062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:19.330082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:19.331092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:19.331623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:19.342628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:19.347675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:19.353839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:19.354705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:19.355493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 139/282] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:19.361099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:19.478101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:19.499603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06ea5c-bdb0-4f30-a9c1-4946b7831e93 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:19.552458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:19.553086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:19.553367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:19.553639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:19.554315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:19.554618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:19.556069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:19.565227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 144/283] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:19.570609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:19.728915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f30fd-4411-4b35-9539-f553d44e1ab8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:19.778228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:19.778872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:19.779185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:19.779456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] PATH_INFO: `/v3/users/4103adfeddda43a9b50ac1ce90de1cdc` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:19.780608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] RBAC: Authorizing `identity:get_user(user_id=4103adfeddda43a9b50ac1ce90de1cdc)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:19.782058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:19.784048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 140/284] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:19 2022] GET /identity/v3/users/4103adfeddda43a9b50ac1ce90de1cdc => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:19.789652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:19.922427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4ddb-ee68-4d58-bad5-76669acac989 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:19.968970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de68365-ab5b-4eb6-a261-25b1e403d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:19.969610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de68365-ab5b-4eb6-a261-25b1e403d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:19.969895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de68365-ab5b-4eb6-a261-25b1e403d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:19.970172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de68365-ab5b-4eb6-a261-25b1e403d8e7 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:19.971377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de68365-ab5b-4eb6-a261-25b1e403d8e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:19.972815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de68365-ab5b-4eb6-a261-25b1e403d8e7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:19.974506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 145/285] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:19 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:19.980414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:20.106907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:20.154841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995ba2e-1ae0-425b-a5ee-2185761bb6a1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:20.155472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a53c8-7ca8-4b15-9764-48eb789e7a23 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:20.155895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a53c8-7ca8-4b15-9764-48eb789e7a23 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:20.156072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a53c8-7ca8-4b15-9764-48eb789e7a23 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:20.157602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 146/286] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:20.162765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:20.163053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:20.163380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:20.196520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:20.197316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:20.197598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:20.197873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/4103adfeddda43a9b50ac1ce90de1cdc/roles/c099a4b10af549a6ab8cff946dca2e5f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:20.199845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f84c0c22d344bdba56d9d3dccd426d0, user_id=4103adfeddda43a9b50ac1ce90de1cdc, role_id=c099a4b10af549a6ab8cff946dca2e5f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:20.201277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:20.204143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:20.205444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:20.215784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:20.277550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 147/287] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:20.285204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 141/288] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:57:19 2022] PUT /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/users/4103adfeddda43a9b50ac1ce90de1cdc/roles/c099a4b10af549a6ab8cff946dca2e5f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:20.631969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:20.771197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4af718-5ae1-4f99-813f-a459660f99ff None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:20.815775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:20.816480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:20.816764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:20.817036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:20.835081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:20.840024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:20.840913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:20.842097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 148/289] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:20.847399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:20.970604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e547d6-b23c-45f9-9787-f0768f455323 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:20.999296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:21.008699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:21.009318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.009598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.009873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.010542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:21.010845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:21.012319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:21.021550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 142/290] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:21.026954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:21.035311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:21.168063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad68d-6331-462b-94d1-6635917a9c9a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:21.212622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:21.213629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.214081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.214530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] PATH_INFO: `/v3/users/93fe86e531d04596bc7e1b101376a212` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.219866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] RBAC: Authorizing `identity:get_user(user_id=93fe86e531d04596bc7e1b101376a212)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:21.221909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:21.223802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 149/291] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:21 2022] GET /identity/v3/users/93fe86e531d04596bc7e1b101376a212 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:21.229434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:21.369832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f4dc1-fe56-4ebb-b904-74121fd8535e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:21.404130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fefc91-bf0d-4c33-b072-57ae00c61986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:21.404748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fefc91-bf0d-4c33-b072-57ae00c61986 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.405030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fefc91-bf0d-4c33-b072-57ae00c61986 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.405305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fefc91-bf0d-4c33-b072-57ae00c61986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.473279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:21.483581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fefc91-bf0d-4c33-b072-57ae00c61986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:21.485210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fefc91-bf0d-4c33-b072-57ae00c61986 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:21.632832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 143/292] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:21.640281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:21.716741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eee17-8e61-46fa-a90c-cf3a481a6dde None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.717475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eee17-8e61-46fa-a90c-cf3a481a6dde None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.717909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eee17-8e61-46fa-a90c-cf3a481a6dde None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.719471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 144/293] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:21.720887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb33425-ef23-4e9f-a8ce-1cb49cc9b71a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.721907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb33425-ef23-4e9f-a8ce-1cb49cc9b71a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.722366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb33425-ef23-4e9f-a8ce-1cb49cc9b71a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.723918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 145/294] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:21.725305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf84e29-cfe4-42d1-85dd-dde747e30792 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.725784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf84e29-cfe4-42d1-85dd-dde747e30792 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.726139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf84e29-cfe4-42d1-85dd-dde747e30792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.742023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:21.747136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4cf84e29-cfe4-42d1-85dd-dde747e30792 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:21.748481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4cf84e29-cfe4-42d1-85dd-dde747e30792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:21.780152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be6d23-a801-4f50-8d36-4788858a2c5b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:21.825678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 146/295] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:21.827434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fa495-8b54-476c-ba4c-ca5129eb6be4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.827665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:21.828156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fa495-8b54-476c-ba4c-ca5129eb6be4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.828363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:21.828652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:21.828926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.829187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fa495-8b54-476c-ba4c-ca5129eb6be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:21.830610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:21.832098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:21.834174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 150/296] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:21 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:21.839442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:21.850029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c03fa495-8b54-476c-ba4c-ca5129eb6be4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:21.851515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c03fa495-8b54-476c-ba4c-ca5129eb6be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:21.871217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:21.977208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9f87-f7db-42f9-9b87-6e036a3c5bfc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.006656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 147/297] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:22.015183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732c71-a5d0-4469-99cd-a753ebc7ec9e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.015777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732c71-a5d0-4469-99cd-a753ebc7ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.016260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732c71-a5d0-4469-99cd-a753ebc7ec9e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.018676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 148/298] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:22.023753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.024041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.024320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.028950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.029568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.029851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.030121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.030856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=93fe86e531d04596bc7e1b101376a212) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:22.031184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:22.032619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:22.042822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 151/299] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:21 2022] GET /identity/v3/role_assignments?user.id=93fe86e531d04596bc7e1b101376a212&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:22.068412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:22.069671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:22.140642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 149/300] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:22.160276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.164245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:22.292761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d88b-b704-467f-a5a9-afec8cf43b79 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.329772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.330956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.331272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.331550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.332693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:22.334131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:22.336224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 152/301] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:22.341554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.423151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:22.437253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.447771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731221a-a83b-4d1f-b3d9-85011dcf0a82 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.495424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.496053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.496330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.496734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] PATH_INFO: `/v3/groups` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.497372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:22.498811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:22.531590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:22.561608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087c5d-c74e-4c92-aa53-19ecf58012e9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.583629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 150/302] 10.222.0.62 () {60 vars in 1134 bytes} [Tue Sep 13 09:57:22 2022] POST /identity/v3/groups => generated 229 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:57:22.584734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.605495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.606582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.607156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.607711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.621508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:22.627804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:22.628807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:22.631916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 153/303] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:22.635319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.725709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e214d4-5f15-4d02-b216-2fd826afeda8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.751727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c72e8-1315-40bb-9b0a-3ecb6dd74be2 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.774655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.775809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.776291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.776779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.785475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:22.788899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:22.793658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:22.797230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.797849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.798374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.798652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.799352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:22.799659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:22.800259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:22.801103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:22.801880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:22.802459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:22.803858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 151/304] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:22.809387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.818479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 154/305] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:22.824708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:22.943957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a59967-fd98-4fbd-b515-df28d581ee6d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.958285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec98350-0da8-4a33-984d-87f65a10625e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:22.981347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:22.982097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:22.982383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:22.982659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] PATH_INFO: `/v3/users/60116d6927ec483fa137342e950110c1` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:22.983935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] RBAC: Authorizing `identity:get_user(user_id=60116d6927ec483fa137342e950110c1)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:22.985438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:22.987589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 155/306] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:22 2022] GET /identity/v3/users/60116d6927ec483fa137342e950110c1 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:22.993277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:23.001726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:23.003052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.003455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.003822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.004643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:23.005103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:23.006609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:23.016063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 152/307] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:23.022904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:23.060915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:23.086669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:23.109297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dce1d-42e7-4821-8ba6-a31eeace8e47 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:23.130216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8f12-d860-411f-b831-2d0fc4cce875 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:23.144809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa7ba6-6774-4cb2-a516-af66da22dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:23.145471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa7ba6-6774-4cb2-a516-af66da22dd53 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.145757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa7ba6-6774-4cb2-a516-af66da22dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.146038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa7ba6-6774-4cb2-a516-af66da22dd53 None admin] PATH_INFO: `/v3/system/users/60116d6927ec483fa137342e950110c1/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.147505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa7ba6-6774-4cb2-a516-af66da22dd53 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=60116d6927ec483fa137342e950110c1, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:23.148895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa7ba6-6774-4cb2-a516-af66da22dd53 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:23.166714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb8bec-3960-47a9-9695-a7040780e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:23.167379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8bec-3960-47a9-9695-a7040780e20e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.167662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8bec-3960-47a9-9695-a7040780e20e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.167933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8bec-3960-47a9-9695-a7040780e20e None admin] PATH_INFO: `/v3/users/64e149d96d0744838355cde2f85a47f3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.181588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb8bec-3960-47a9-9695-a7040780e20e None admin] RBAC: Authorizing `identity:get_user(user_id=64e149d96d0744838355cde2f85a47f3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:23.183026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb8bec-3960-47a9-9695-a7040780e20e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:23.184699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 153/308] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:23 2022] GET /identity/v3/users/64e149d96d0744838355cde2f85a47f3 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:23.185808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdb949-4192-44a5-9712-749db34cc13a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.186093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdb949-4192-44a5-9712-749db34cc13a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.186373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdb949-4192-44a5-9712-749db34cc13a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.187755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 154/309] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:23.191223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdb949-4192-44a5-9712-749db34cc13a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:23.234109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 156/310] 10.222.0.62 () {58 vars in 1334 bytes} [Tue Sep 13 09:57:22 2022] PUT /identity/v3/system/users/60116d6927ec483fa137342e950110c1/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:57:23.236987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.237273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.237552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.257764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:23.258986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:23.317700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdb949-4192-44a5-9712-749db34cc13a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:23.338650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:23.349538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 157/311] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:23.364951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:23.365624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.365954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.366226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.366955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] RBAC: Adding query filter params (role.id=c099a4b10af549a6ab8cff946dca2e5f, user.id=64e149d96d0744838355cde2f85a47f3, scope.system=all) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:23.367308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:23.368865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:23.376000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 155/312] 10.222.0.62 () {56 vars in 1308 bytes} [Tue Sep 13 09:57:23 2022] GET /identity/v3/role_assignments?user.id=64e149d96d0744838355cde2f85a47f3&scope.system=all&role.id=c099a4b10af549a6ab8cff946dca2e5f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:23.627939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:23.689734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:23.735005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30efb7-7b8e-482c-b911-5aa0b81ddba0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:23.771394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:23.772281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.772562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.772837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.784349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:23.789390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:23.790257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:23.791339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:23.793832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 158/313] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:23.800125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:23.903678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812944-90de-4c49-b26d-747ead8d2e68 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:23.937873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:23.938495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:23.938778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:23.939053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:23.939777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:23.940084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:23.941518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:23.950265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 156/314] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:23.956545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:24.060267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aad4d-c41d-4c5f-a6c4-5c8888da7b52 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:24.076701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:24.094864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:24.095550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:24.096430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:24.096713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] PATH_INFO: `/v3/users/4103adfeddda43a9b50ac1ce90de1cdc` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:24.097861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] RBAC: Authorizing `identity:get_user(user_id=4103adfeddda43a9b50ac1ce90de1cdc)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:24.099367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:24.101281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 159/315] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:23 2022] GET /identity/v3/users/4103adfeddda43a9b50ac1ce90de1cdc => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:24.106871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:24.160073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:24.223309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ca058-400b-4d5d-aeb2-b2c1505a7eb6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:24.257498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:24.258105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:24.258386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:24.258658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:24.328707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:24.330838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:24.391077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:24.436851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 157/316] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:24.442625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:24.545476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df9f0-4b66-47f0-ad6b-09116976deed None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:24.581688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569b93-60a2-4ae6-a55e-7b11ec8a84af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:24.582332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569b93-60a2-4ae6-a55e-7b11ec8a84af None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:24.582624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569b93-60a2-4ae6-a55e-7b11ec8a84af None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:24.582900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569b93-60a2-4ae6-a55e-7b11ec8a84af None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:24.584066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2569b93-60a2-4ae6-a55e-7b11ec8a84af None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:24.585601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2569b93-60a2-4ae6-a55e-7b11ec8a84af None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:24.587570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 160/317] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:24 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:24.593421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:24.698023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578bf4-c8ca-4b84-a3f1-cabb186a84eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:24.737135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:24.737739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:24.738069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:24.738343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:24.739063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] RBAC: Adding query filter params (role.id=c099a4b10af549a6ab8cff946dca2e5f, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0, user.id=4103adfeddda43a9b50ac1ce90de1cdc) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:24.739403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:24.740821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe453d-f154-4e4e-9b22-15bb8b485f26 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:24.746879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 158/318] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:24 2022] GET /identity/v3/role_assignments?user.id=4103adfeddda43a9b50ac1ce90de1cdc&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=c099a4b10af549a6ab8cff946dca2e5f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:24.967845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf21c-e679-4422-b70d-7cc34dfb1360 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:24.968055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf21c-e679-4422-b70d-7cc34dfb1360 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:24.968332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf21c-e679-4422-b70d-7cc34dfb1360 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:24.969925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 161/319] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:24.976087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:24.976372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:24.976647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:24.997303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:24.998800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:25.001282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:25.068268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 159/320] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:25.343404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdf21c-e679-4422-b70d-7cc34dfb1360 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:25.404133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:25.455486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdf21c-e679-4422-b70d-7cc34dfb1360 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:25.489076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1b7a7-7ce2-4178-9a87-dbf1f8599aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:25.489765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1b7a7-7ce2-4178-9a87-dbf1f8599aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:25.490048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1b7a7-7ce2-4178-9a87-dbf1f8599aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:25.490323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1b7a7-7ce2-4178-9a87-dbf1f8599aa8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:25.491510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1b7a7-7ce2-4178-9a87-dbf1f8599aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:25.492961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1b7a7-7ce2-4178-9a87-dbf1f8599aa8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:25.494693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 162/321] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:25.500278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:25.517402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2875589-1233-4de2-8b23-6f32dd472947 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:25.517755 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2875589-1233-4de2-8b23-6f32dd472947 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:25.517971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2875589-1233-4de2-8b23-6f32dd472947 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:25.519580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 163/322] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:25.525409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731c60a-66a4-441d-bd4c-1f684339c6d1 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:25.525708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731c60a-66a4-441d-bd4c-1f684339c6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:25.525969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731c60a-66a4-441d-bd4c-1f684339c6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:25.546263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7731c60a-66a4-441d-bd4c-1f684339c6d1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:25.547598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7731c60a-66a4-441d-bd4c-1f684339c6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:25.610415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee137c-91b3-4556-bdca-076c14d39f06 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:25.621380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 164/323] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:25.635468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9cf08-1753-4648-a01d-51c33f5fe644 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:25.635758 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9cf08-1753-4648-a01d-51c33f5fe644 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:25.636035 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9cf08-1753-4648-a01d-51c33f5fe644 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:25.637348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 165/324] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:25.642049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:25.642335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:25.642645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:25.646969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:25.647606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:25.647885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:25.648154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] PATH_INFO: `/v3/groups` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:25.648794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:25.650227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:25.661263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:25.662452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:25.708515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:25.728789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 166/325] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:25.781274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:25.787442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 160/326] 10.222.0.62 () {60 vars in 1134 bytes} [Tue Sep 13 09:57:25 2022] POST /identity/v3/groups => generated 226 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:57:25.890132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:25.999749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.003752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e68d4-0810-4dab-b272-3f8551c3cb82 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.040343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.041233 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.041518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.041794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.059213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:26.064163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:26.068051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:26.068838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 167/327] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:26.074611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.124246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:26.126166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f081c87-2be5-43e0-bb5d-91c20ca373e0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.172657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.173690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.174217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.174690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.187139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:26.188879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812df76b-af50-415d-b2e4-7d37dc216387 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:26.192538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:26.193393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:26.194177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 161/328] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:26.201345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.238117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.239059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.239595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.240067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.240935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:26.241439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:26.243139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:26.253099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 168/329] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:26.259142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.269547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:26.321662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdcc65-6a53-4662-8ea2-44339328743d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.362857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.363517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.363777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.364046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.364886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:26.365188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:26.367246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:26.376201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 162/330] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:26.390469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea7a39-64d4-44ac-9ed1-58be79dafb2e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.391152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.393654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:26.444639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.445216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.445500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.445767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] PATH_INFO: `/v3/users/60116d6927ec483fa137342e950110c1` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.447021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] RBAC: Authorizing `identity:get_user(user_id=60116d6927ec483fa137342e950110c1)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:26.448528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:26.450419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 169/331] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:26 2022] GET /identity/v3/users/60116d6927ec483fa137342e950110c1 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:26.456605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.517194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1781fef-47a6-4339-a274-ff819806c217 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.545063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:26.563141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.563805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.564855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.565135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] PATH_INFO: `/v3/users/64e149d96d0744838355cde2f85a47f3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.567081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] RBAC: Authorizing `identity:get_user(user_id=64e149d96d0744838355cde2f85a47f3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:26.568766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:26.570953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 163/332] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:26 2022] GET /identity/v3/users/64e149d96d0744838355cde2f85a47f3 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:26.571661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1269f5c-ad99-4c67-a270-8de1fa7eacbd None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.577017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:26.608322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.608949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.609211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.609491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.610219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] RBAC: Adding query filter params (role.id=28d0ae899af34488a7f267d899b9aff7, user.id=60116d6927ec483fa137342e950110c1, scope.system=all) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:26.610503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:26.612011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae7344-454a-497b-81e8-38ff348a1c0d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:26.624424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 170/333] 10.222.0.62 () {56 vars in 1308 bytes} [Tue Sep 13 09:57:26 2022] GET /identity/v3/role_assignments?user.id=60116d6927ec483fa137342e950110c1&scope.system=all&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:26.727591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639309a0-59f0-4d69-912b-56dac65ade0a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:26.770744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da9f71-e996-4c4d-84d2-ffb31b858ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:26.771445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9f71-e996-4c4d-84d2-ffb31b858ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:26.771729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9f71-e996-4c4d-84d2-ffb31b858ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:26.772006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9f71-e996-4c4d-84d2-ffb31b858ab3 None admin] PATH_INFO: `/v3/system/users/64e149d96d0744838355cde2f85a47f3/roles/c099a4b10af549a6ab8cff946dca2e5f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:26.773501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9f71-e996-4c4d-84d2-ffb31b858ab3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=64e149d96d0744838355cde2f85a47f3, role_id=c099a4b10af549a6ab8cff946dca2e5f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:26.774915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9f71-e996-4c4d-84d2-ffb31b858ab3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:26.838342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 164/334] 10.222.0.62 () {58 vars in 1334 bytes} [Tue Sep 13 09:57:26 2022] PUT /identity/v3/system/users/64e149d96d0744838355cde2f85a47f3/roles/c099a4b10af549a6ab8cff946dca2e5f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:57:28.293401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe389c1-4cf7-4df4-8119-8380e627916c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:28.293908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe389c1-4cf7-4df4-8119-8380e627916c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:28.293981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe389c1-4cf7-4df4-8119-8380e627916c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:28.295559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 171/335] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:28.301634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9f9f7-2ff9-4d0e-8807-3b5f63885a99 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:28.302062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9f9f7-2ff9-4d0e-8807-3b5f63885a99 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:28.302235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9f9f7-2ff9-4d0e-8807-3b5f63885a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:28.328167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a4d9f9f7-2ff9-4d0e-8807-3b5f63885a99 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:28.329420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a4d9f9f7-2ff9-4d0e-8807-3b5f63885a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:28.404215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:28.425472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 165/336] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:28.742827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e1db6-cea7-4a85-ad2f-dc071146fe13 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:28.743206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e1db6-cea7-4a85-ad2f-dc071146fe13 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:28.743405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e1db6-cea7-4a85-ad2f-dc071146fe13 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:28.744990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 172/337] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:28.761837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:28.762125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:28.762398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:28.788117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:28.789308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:28.867524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 166/338] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:28.950443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e1db6-cea7-4a85-ad2f-dc071146fe13 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:28.985669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:29.076417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e1db6-cea7-4a85-ad2f-dc071146fe13 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:29.116960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:29.117592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.117872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.118135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.132047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:29.137453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:29.138321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:29.139110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 173/339] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:29.144877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:29.253935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:29.264952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a4c61-5194-47d5-8b7e-ce7fe0ee8630 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:29.299369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:29.338473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:29.339071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.339320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.339578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.340252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:29.340553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:29.341970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:29.353949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 167/340] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:29.361176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:29.463194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:29.497817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345387ae-b54e-461c-9449-9d465a626dc1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:29.514955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751d24f-2f02-448c-942c-b036e4137075 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:29.570253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310777a-3622-4b33-b48b-3216234c43e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:29.571146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310777a-3622-4b33-b48b-3216234c43e6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.571706 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310777a-3622-4b33-b48b-3216234c43e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.572116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310777a-3622-4b33-b48b-3216234c43e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.590442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c936b7c-659d-4f30-b63d-0a58a8153512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:29.591377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c936b7c-659d-4f30-b63d-0a58a8153512 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.591800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c936b7c-659d-4f30-b63d-0a58a8153512 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.592212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c936b7c-659d-4f30-b63d-0a58a8153512 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.625751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c936b7c-659d-4f30-b63d-0a58a8153512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6c936b7c-659d-4f30-b63d-0a58a8153512 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:29.636439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:29.637559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:29.638552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:29.639218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:29.641287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 174/341] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:29.643299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0248b18-fc2b-4396-a5a3-3779d1869a67 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.643741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0248b18-fc2b-4396-a5a3-3779d1869a67 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.644146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0248b18-fc2b-4396-a5a3-3779d1869a67 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.645914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 175/342] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:29.651757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0248b18-fc2b-4396-a5a3-3779d1869a67 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:29.669392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310777a-3622-4b33-b48b-3216234c43e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:29.671103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310777a-3622-4b33-b48b-3216234c43e6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:29.828721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 168/343] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:29.831252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.831768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.832258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.846631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0248b18-fc2b-4396-a5a3-3779d1869a67 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:29.853737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:29.855293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:29.905018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:29.905999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:29.907177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:29.907650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:29.908966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:29.909785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:29.911528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:29.928477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 176/344] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:29.938387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:29.959181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:29.992412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 169/345] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:29.993918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.003542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:30.121679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b12f4-6b61-4337-9a82-911a46abd91f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:30.180667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f014c0-93d3-4a5f-a721-361c2a305db9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:30.192319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50889fa2-b972-44fd-9d45-662ed495c1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:30.192967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50889fa2-b972-44fd-9d45-662ed495c1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.193250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50889fa2-b972-44fd-9d45-662ed495c1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.193526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50889fa2-b972-44fd-9d45-662ed495c1c0 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.194693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50889fa2-b972-44fd-9d45-662ed495c1c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:30.196186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50889fa2-b972-44fd-9d45-662ed495c1c0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:30.197825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 177/346] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:29 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:30.203885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7cdfe-185d-4b44-a7e9-f035767a6d42 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.204175 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7cdfe-185d-4b44-a7e9-f035767a6d42 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.204450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7cdfe-185d-4b44-a7e9-f035767a6d42 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.206152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 178/347] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:30.207021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7cdfe-185d-4b44-a7e9-f035767a6d42 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.247831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf885c9-7ede-43ab-b5ae-39a822fda7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:30.248723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf885c9-7ede-43ab-b5ae-39a822fda7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.249145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf885c9-7ede-43ab-b5ae-39a822fda7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.249545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf885c9-7ede-43ab-b5ae-39a822fda7b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.355633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf885c9-7ede-43ab-b5ae-39a822fda7b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:30.357155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf885c9-7ede-43ab-b5ae-39a822fda7b4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:30.365244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:30.396428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:30.434591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7cdfe-185d-4b44-a7e9-f035767a6d42 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:30.477183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f398052-a148-4b98-bdb4-42ed8cfbf680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:30.477869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f398052-a148-4b98-bdb4-42ed8cfbf680 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.478104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f398052-a148-4b98-bdb4-42ed8cfbf680 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.478381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f398052-a148-4b98-bdb4-42ed8cfbf680 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.487388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f398052-a148-4b98-bdb4-42ed8cfbf680 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:30.488499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f398052-a148-4b98-bdb4-42ed8cfbf680 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:30.490586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 179/348] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:30 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:30.492056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.492596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.493089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.510908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 170/349] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:30.513341 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbc90d-524c-44c9-b947-6d05716dec7a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.513770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbc90d-524c-44c9-b947-6d05716dec7a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.514083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbc90d-524c-44c9-b947-6d05716dec7a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.514372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:30.518540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 171/350] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:30.518674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:30.518749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbc90d-524c-44c9-b947-6d05716dec7a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.612396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 180/351] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:30.613586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.648697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbc90d-524c-44c9-b947-6d05716dec7a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:30.651078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:30.688777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffe0c2-1804-4f3f-93e1-4ba1279b7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:30.689974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffe0c2-1804-4f3f-93e1-4ba1279b7ebf None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.690664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffe0c2-1804-4f3f-93e1-4ba1279b7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.690664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffe0c2-1804-4f3f-93e1-4ba1279b7ebf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.702177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffe0c2-1804-4f3f-93e1-4ba1279b7ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d2ffe0c2-1804-4f3f-93e1-4ba1279b7ebf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:30.707895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:30.708797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:30.709607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 172/352] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:30.710736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.710830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.711053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.723413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:30.730900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:30.732137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:30.761674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79c33f-1e85-4f2f-b7ef-a73db30a1793 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:30.799730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:30.800345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.800681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.801055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.801943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:30.802259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:30.803811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:30.806403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 173/353] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:30.807323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.811795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 181/354] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:30 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:30.812655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.910129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:30.917496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983916c-c7c0-4671-945b-b2cac4a75fc1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:30.956058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:30.956818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:30.957099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:30.957370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:30.958633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:30.960154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:30.966656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 174/355] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:30 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:30.968170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:30.999352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbc339-9349-4a79-b9ae-81ef239c1e70 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.072016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.073079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.073581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.074067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.075025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:31.075576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:31.077242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:31.080265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:31.083077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3af4ee-538a-4e19-bed7-cdef7d0f437a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.088791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 182/356] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:31.094802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.123809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.124468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.124748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.125023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.127380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:31.128424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:31.132928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 175/357] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:30 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:31.135842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.175503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:31.227966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719d14a-8c0c-4f88-b7bb-40dd310035ee None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.276053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1122a12-da44-4e87-a1b5-9a1290dc90cc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.283393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3060c45-6476-4b66-9960-d5c20a08dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.284281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3060c45-6476-4b66-9960-d5c20a08dd06 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.284754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3060c45-6476-4b66-9960-d5c20a08dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.285238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3060c45-6476-4b66-9960-d5c20a08dd06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.332111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.333017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.333601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.334093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.335097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:31.335640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:31.338358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:31.346856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 176/358] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:31.348059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.376638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:31.384481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3060c45-6476-4b66-9960-d5c20a08dd06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:31.387627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3060c45-6476-4b66-9960-d5c20a08dd06 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:31.486748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:31.496610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87153-4070-4800-9c2f-9ff88c2bf66a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.512885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 183/359] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:31.514844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e9ec-d217-4a37-8cc5-dc0baf3d3113 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.515371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e9ec-d217-4a37-8cc5-dc0baf3d3113 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.515922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e9ec-d217-4a37-8cc5-dc0baf3d3113 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.517789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 184/360] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:31.518768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e9ec-d217-4a37-8cc5-dc0baf3d3113 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.550185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.551129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.551642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.552059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.565962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:31.572562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:31.573426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 177/361] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:31.574248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.648052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:31.653512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e9ec-d217-4a37-8cc5-dc0baf3d3113 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.697361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70317-ede8-4464-9a6f-c673681524af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.697993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70317-ede8-4464-9a6f-c673681524af None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.698270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70317-ede8-4464-9a6f-c673681524af None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.698536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70317-ede8-4464-9a6f-c673681524af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.709659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d70317-ede8-4464-9a6f-c673681524af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:31.713983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13b3d-92f1-4044-ba37-221ac6db00e0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-55d70317-ede8-4464-9a6f-c673681524af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:31.715735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:31.716632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:31.717429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 185/362] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:31.723188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.723188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.723378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.744838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:31.746109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:31.753976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.754615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.754899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.755189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.756883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:31.758868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:31.761202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 178/363] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:31.762292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.804615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:31.837304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 186/364] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:31.840588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:31.898720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe33997-8789-4f8b-95d1-eb3b6e8daefa None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:31.947903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:31.948519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:31.948795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:31.949065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:31.950116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:31.952136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:31.953697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:31.974903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 179/365] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:31.975821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.032402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.058265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9e6a3-49ad-4afd-ae38-612a609d3f7b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.133913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.134954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.135471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.135976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.136843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:32.137347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.138951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.159429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 187/366] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:32.164471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.165912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.189705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca356998-b6ca-4d02-b1d2-02e9e814a034 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.263033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.263722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.263998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.264267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.265380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.267016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.271053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 180/367] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:31 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:32.271053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.314543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc183f-464c-4b3d-9ddf-0a435d62c7e9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.354598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.355438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.355905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.356192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.379747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.422674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a420b-97ff-426f-9a65-d5e05dfe7960 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.442084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.444177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.463280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.464070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.464574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.464945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] PATH_INFO: `/v3/users/64e149d96d0744838355cde2f85a47f3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.466885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] RBAC: Authorizing `identity:get_user(user_id=64e149d96d0744838355cde2f85a47f3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.468752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.471581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 181/368] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/users/64e149d96d0744838355cde2f85a47f3 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:32.472664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.506973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.585979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 188/369] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:32.587114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.615345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4201a9c-e70f-4c78-9bd1-dfbbfd3387b4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.644724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.661362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.662367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.662836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.663324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.677553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:32.683718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:32.684572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:32.685349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:32.688551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 182/370] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:32.690112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.710308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40afcc-15d4-4f92-bc10-742812264399 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.751123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.752287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.752566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.752918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.753555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:32.753862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.755410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.763387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 189/371] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:32.764212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.793834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.818383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5d56c-ac05-476f-8f6e-0d7698bfe489 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.896696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.897251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.897544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.897806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.898548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] RBAC: Adding query filter params (role.id=c099a4b10af549a6ab8cff946dca2e5f, user.id=64e149d96d0744838355cde2f85a47f3, scope.system=all) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:32.898864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.900388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.912900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19332bc7-47ce-4b2b-8127-f9608e3d4fd6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:32.933916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:32.934635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 183/372] 10.222.0.62 () {56 vars in 1308 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/role_assignments?user.id=64e149d96d0744838355cde2f85a47f3&scope.system=all&role.id=c099a4b10af549a6ab8cff946dca2e5f => generated 510 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:32.935601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:32.978115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:32.979193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:32.979937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:32.987669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:32.989069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:32.990574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:32.993393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 190/373] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:32.997740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:33.086171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e4e2-7161-4ad6-aae1-5215b56729b5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:33.126208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:33.127784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.128273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.128741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.129641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:33.130168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:33.132322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:33.139392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:33.145164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 184/374] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:33.151772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:33.186364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd20566-6d36-4750-9228-9dd9901d2dfe None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:33.234976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:33.235803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.236759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.237600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.239910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:33.242284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:33.244973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 191/375] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:32 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:33.250300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:33.276930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379da6a0-bb58-4320-b748-e6023e8f5355 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:33.315402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:33.321739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:33.322667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.323091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.323705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.380498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54aed9-af5a-420c-8a51-744fce325b02 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:33.417587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:33.423075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:33.425290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:33.427432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334673e-b392-4018-bafc-02502a514277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:33.429072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334673e-b392-4018-bafc-02502a514277 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.429404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334673e-b392-4018-bafc-02502a514277 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.429763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334673e-b392-4018-bafc-02502a514277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.430573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334673e-b392-4018-bafc-02502a514277 None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:33.430959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334673e-b392-4018-bafc-02502a514277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:33.432498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334673e-b392-4018-bafc-02502a514277 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:33.441509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 192/376] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:33 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:33.552679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 185/377] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:33.558596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334673e-b392-4018-bafc-02502a514277 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:33.683182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334673e-b392-4018-bafc-02502a514277 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:33.699397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:33.727882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e80e3a-27b1-4ec5-abf1-8c1927d62d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:33.728795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e80e3a-27b1-4ec5-abf1-8c1927d62d1c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.729267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e80e3a-27b1-4ec5-abf1-8c1927d62d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.729718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e80e3a-27b1-4ec5-abf1-8c1927d62d1c None admin] PATH_INFO: `/v3/projects/60981a9be15c4b04b78b97afc4285aa0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.731093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e80e3a-27b1-4ec5-abf1-8c1927d62d1c None admin] RBAC: Authorizing `identity:get_project(project_id=60981a9be15c4b04b78b97afc4285aa0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:33.732735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e80e3a-27b1-4ec5-abf1-8c1927d62d1c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:33.734870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 193/378] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:33 2022] GET /identity/v3/projects/60981a9be15c4b04b78b97afc4285aa0 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:33.741359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:33.872152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:33.875813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd65a4-8981-4dfd-9085-0dae29885eb7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:33.888168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba057-00be-407e-a517-415b96bb976d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.890524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba057-00be-407e-a517-415b96bb976d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.890524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba057-00be-407e-a517-415b96bb976d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.890524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 194/379] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:33.900637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a51f5-aa41-43c8-bd79-9a4fe58242b5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.901226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a51f5-aa41-43c8-bd79-9a4fe58242b5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.901699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a51f5-aa41-43c8-bd79-9a4fe58242b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.927460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:33.928486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:33.928970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:33.929452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] PATH_INFO: `/v3/groups/36530794718f437d8addf0f0efaea6a6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:33.935837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c69a51f5-aa41-43c8-bd79-9a4fe58242b5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:33.938929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c69a51f5-aa41-43c8-bd79-9a4fe58242b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:33.945216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] RBAC: Authorizing `identity:get_group(group_id=36530794718f437d8addf0f0efaea6a6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:33.946933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:33.949650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 186/380] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:33 2022] GET /identity/v3/groups/36530794718f437d8addf0f0efaea6a6 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:33.964143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:34.053032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 195/381] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:34.113205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e216a5-c00a-4063-8d66-b249ea4ea47b None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:34.113986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e216a5-c00a-4063-8d66-b249ea4ea47b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:34.114270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e216a5-c00a-4063-8d66-b249ea4ea47b None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:34.116116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 196/382] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:34.127287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:34.127566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:34.127839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:34.146190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9b9b3-67a6-4107-9d6d-229113031979 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:34.160193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:34.161416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:34.184337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:34.185300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:34.185784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:34.186349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:34.187456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] RBAC: Adding query filter params (group.id=36530794718f437d8addf0f0efaea6a6, role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=60981a9be15c4b04b78b97afc4285aa0) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:34.187973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:34.189629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:34.196127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 187/383] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:33 2022] GET /identity/v3/role_assignments?group.id=36530794718f437d8addf0f0efaea6a6&scope.project.id=60981a9be15c4b04b78b97afc4285aa0&role.id=435125566e954d20a3b8273b7c806b98 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:34.264860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 197/384] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 09:57:34.421830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:34.453166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:34.531083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0812d75-d113-41f3-ae03-e8f851a50d0d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:34.569292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:34.570693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:34.571199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:34.571710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:34.582702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:34.588014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:34.588924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:34.589749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:34.592344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 188/385] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:34.599007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:34.664869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:34.729055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbe9c8-eb93-4a23-83dc-d4da55ff2770 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:34.761672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:34.779822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:34.780447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:34.780724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:34.781001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:34.781784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:34.782093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:34.783539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:34.792503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 198/386] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:34.799248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:34.911258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00cece-5d18-4aa1-a4fc-9417aeef1409 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:34.935548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:34.947041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:34.947735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:34.948018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:34.948299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:34.949498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:34.951075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:34.952825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 199/387] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:34 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:34.958538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:35.006655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:35.037359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd948fa-e8d7-4bb0-a8cb-7dad4aceb5c7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:35.076927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1878270-b195-45a1-9ccc-ab81be2e273f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:35.114206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:35.114860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:35.115149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:35.115455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:35.115957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:35.116612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:35.116717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:35.116923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:35.117202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:35.118072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:35.120297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 200/388] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:34 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:35.125670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:35.146255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:35.151195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:35.152046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:35.152869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:35.155108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 189/389] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:35.160101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:35.191544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:35.240901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7dfbad-b59f-4c8f-a3a0-8c68cc5aa414 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:35.283119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:35.283717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:35.284003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:35.284280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/groups/b2f09a67e6fa4f54a813949e51f78473/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:35.286340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] RBAC: Authorizing `identity:create_grant(project_id=5a44c7b4a430429e94e14dfb7e2c81a5, group_id=b2f09a67e6fa4f54a813949e51f78473, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:35.287963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:35.347525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ee8ca-fccc-4f2b-aa06-0615ebb44a30 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:35.370038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 201/390] 10.222.0.62 () {58 vars in 1442 bytes} [Tue Sep 13 09:57:35 2022] PUT /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/groups/b2f09a67e6fa4f54a813949e51f78473/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:35.395516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:35.396149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:35.396424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:35.396794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:35.397479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:35.397786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:35.399298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:35.412948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 190/391] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:35.414288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:35.574555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:35.577847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358a9d-6e94-4acb-b809-6a02122b4deb None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:35.615809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:35.616827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:35.617108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:35.617379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:35.618495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:35.619940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:35.621799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 202/392] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:35 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:35.627828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:35.641633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:35.743076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186566-6324-4942-911f-0f2a041b9956 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:35.778267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091648f4-c8cc-4987-ae61-ad692de556ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:35.778996 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091648f4-c8cc-4987-ae61-ad692de556ee None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:35.779315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091648f4-c8cc-4987-ae61-ad692de556ee None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:35.779587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091648f4-c8cc-4987-ae61-ad692de556ee None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:35.780824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091648f4-c8cc-4987-ae61-ad692de556ee None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:35.782523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091648f4-c8cc-4987-ae61-ad692de556ee None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:35.785401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 191/393] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:35 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:35.791113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:35.948111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca0201-691d-471a-a89d-4698ee46a3de None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:36.006732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dedec2-a6db-4a4e-87c9-222b8c5a5110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:36.007445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dedec2-a6db-4a4e-87c9-222b8c5a5110 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.007731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dedec2-a6db-4a4e-87c9-222b8c5a5110 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.008007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dedec2-a6db-4a4e-87c9-222b8c5a5110 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/groups/b2f09a67e6fa4f54a813949e51f78473/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.009111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7030ca-079d-4fd5-ab6d-e983b61d87b5 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.009588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7030ca-079d-4fd5-ab6d-e983b61d87b5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.010061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7030ca-079d-4fd5-ab6d-e983b61d87b5 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.010170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dedec2-a6db-4a4e-87c9-222b8c5a5110 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a44c7b4a430429e94e14dfb7e2c81a5, group_id=b2f09a67e6fa4f54a813949e51f78473, role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:36.011649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dedec2-a6db-4a4e-87c9-222b8c5a5110 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:36.012184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 192/394] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:36.018137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5d618-0dba-41fc-a793-7e0bd849c8e2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.018686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5d618-0dba-41fc-a793-7e0bd849c8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.019201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5d618-0dba-41fc-a793-7e0bd849c8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.046805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5ae5d618-0dba-41fc-a793-7e0bd849c8e2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:36.048070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5ae5d618-0dba-41fc-a793-7e0bd849c8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:36.097875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 203/395] 10.222.0.62 () {58 vars in 1442 bytes} [Tue Sep 13 09:57:35 2022] PUT /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5/groups/b2f09a67e6fa4f54a813949e51f78473/roles/98057a16b9eb4d96a1931655b2cd15be => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:57:36.133313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 193/396] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:36.366633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a9f87-4358-432a-a2a6-19ffe6faf3ee None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.366914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a9f87-4358-432a-a2a6-19ffe6faf3ee None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.367275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a9f87-4358-432a-a2a6-19ffe6faf3ee None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.368714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 204/397] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:36.376078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.376078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.377011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.405730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:36.406948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:36.409079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:36.482937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 194/398] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:36.605013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a9f87-4358-432a-a2a6-19ffe6faf3ee None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:36.637568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:36.737122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a9f87-4358-432a-a2a6-19ffe6faf3ee None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:36.774910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:36.775953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.776427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.776903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.792654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:36.799088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:36.799965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:36.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:36.803314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 205/399] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:36.808916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:36.813869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:36.912925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:36.925554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62b14-5b6a-47b9-a316-64a992db29d0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:36.962027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:36.962871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:36.963156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:36.963455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:36.974879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:36.979933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:36.980907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:36.981697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 206/400] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:36.987033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.022072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe049d-cfad-4d9f-a819-7946382fcbf2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.107280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da40173-0134-4448-84f0-3c97f825df8d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.116478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.117386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.117873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.118328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.119213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:37.119709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.121340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.134413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:37.143638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 195/401] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:37.148363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.151384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.151384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.151565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.151646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.151646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:37.151646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.153013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.162753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 207/402] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:37.171367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.186327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:37.281611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3cbfe-8804-4e70-8a0e-5af5b40ce20a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.302299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91f8b9-6e64-4883-a85b-cc2b4de9eb49 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.319936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.320697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.321053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.321335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.322585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.324082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.326169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 208/403] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:37 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:37.333324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.362886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8819f-5443-4cb1-a75c-41d35ae52f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.363771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8819f-5443-4cb1-a75c-41d35ae52f37 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.364055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8819f-5443-4cb1-a75c-41d35ae52f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.364327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8819f-5443-4cb1-a75c-41d35ae52f37 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.367466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8819f-5443-4cb1-a75c-41d35ae52f37 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.369638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8819f-5443-4cb1-a75c-41d35ae52f37 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.371875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 196/404] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:37 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:37.373776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0dd0b-e06e-47dd-96ae-6c8f997b473d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.373776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0dd0b-e06e-47dd-96ae-6c8f997b473d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.373927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0dd0b-e06e-47dd-96ae-6c8f997b473d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.375645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 197/405] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:37.380713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0dd0b-e06e-47dd-96ae-6c8f997b473d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.442481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:37.453008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:37.462835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb6c87-cdd7-4880-8cc5-91e2bc82ca1a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.489612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0dd0b-e06e-47dd-96ae-6c8f997b473d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.499280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dc70e-f70c-4c62-86d6-2839b3d6ba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.500104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dc70e-f70c-4c62-86d6-2839b3d6ba6b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.500568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dc70e-f70c-4c62-86d6-2839b3d6ba6b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.500979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dc70e-f70c-4c62-86d6-2839b3d6ba6b None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.502344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dc70e-f70c-4c62-86d6-2839b3d6ba6b None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.503954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dc70e-f70c-4c62-86d6-2839b3d6ba6b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.510654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 209/406] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:37 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:37.513006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.513523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.513958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.525968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.526642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.526973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.527287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.528536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.530102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.532182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 198/407] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:37 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:37.533829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.534949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:37.536452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:37.607694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 210/408] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:37.608816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:37.663209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f32d50-d5fb-412b-9a5a-1036674a7b52 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.695529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:37.720398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f14ef-c5ba-4046-9778-04c6fb43b57f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:37.729805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.730764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.730881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.731074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/groups/b2f09a67e6fa4f54a813949e51f78473/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.735769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] RBAC: Authorizing `identity:create_grant(project_id=4f84c0c22d344bdba56d9d3dccd426d0, group_id=b2f09a67e6fa4f54a813949e51f78473, role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.737695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.741175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:37.759892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:37.760762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:37.761427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:37.761737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/groups/b2f09a67e6fa4f54a813949e51f78473/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:37.763879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f84c0c22d344bdba56d9d3dccd426d0, group_id=b2f09a67e6fa4f54a813949e51f78473, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:37.765464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:37.834283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 211/409] 10.222.0.62 () {58 vars in 1442 bytes} [Tue Sep 13 09:57:37 2022] PUT /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/groups/b2f09a67e6fa4f54a813949e51f78473/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:37.839285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 199/410] 10.222.0.62 () {58 vars in 1442 bytes} [Tue Sep 13 09:57:37 2022] PUT /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0/groups/b2f09a67e6fa4f54a813949e51f78473/roles/98057a16b9eb4d96a1931655b2cd15be => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:57:38.071277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:38.209229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7131ebc-5bd1-4b39-ae79-5aaab40c5332 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:38.264380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91883e7d-f395-4f5e-86b2-83b98a3a2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:38.265368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883e7d-f395-4f5e-86b2-83b98a3a2b50 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.265872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883e7d-f395-4f5e-86b2-83b98a3a2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.266334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883e7d-f395-4f5e-86b2-83b98a3a2b50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.272599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:38.281015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91883e7d-f395-4f5e-86b2-83b98a3a2b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-91883e7d-f395-4f5e-86b2-83b98a3a2b50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:38.286013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:38.286906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:38.287708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 212/411] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:38.296290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:38.393816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222419f-ce6e-4b76-a733-8a77520b5ce7 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.394206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222419f-ce6e-4b76-a733-8a77520b5ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.394481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222419f-ce6e-4b76-a733-8a77520b5ce7 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.397028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 213/412] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:38.403089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a71b-2554-44d7-9345-7fd398da5e75 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.403408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a71b-2554-44d7-9345-7fd398da5e75 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.403935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a71b-2554-44d7-9345-7fd398da5e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.438379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-67e2a71b-2554-44d7-9345-7fd398da5e75 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:38.440167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-67e2a71b-2554-44d7-9345-7fd398da5e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:38.469259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea8c3-4308-4196-9acd-997ac02c89ba None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:38.513135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:38.515319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.516853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.517687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.519825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:38.520331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:38.522967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:38.541710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 200/413] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:38.549590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:38.587232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 214/414] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:38.652838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:38.709885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da81f-ca3f-4dec-afd7-baab2977fabe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:38.763919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:38.764558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.764838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.765112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] PATH_INFO: `/v3/groups/36530794718f437d8addf0f0efaea6a6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.771848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] RBAC: Authorizing `identity:get_group(group_id=36530794718f437d8addf0f0efaea6a6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:38.773313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:38.774913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 201/415] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:38 2022] GET /identity/v3/groups/36530794718f437d8addf0f0efaea6a6 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:38.780795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82082-191f-45e7-83d6-01f6ee3a8059 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.781085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82082-191f-45e7-83d6-01f6ee3a8059 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.781363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82082-191f-45e7-83d6-01f6ee3a8059 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.783248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 215/416] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:38.784117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:38.798850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.799124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.799354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.826876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:38.828108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:38.937184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 216/417] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:38.937994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:38.953303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263cbed-d4d2-4c0d-9019-5733f1b1e07b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:38.986528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:38.995952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:38.996837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:38.997118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:38.997792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] PATH_INFO: `/v3/projects/60981a9be15c4b04b78b97afc4285aa0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:38.998926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] RBAC: Authorizing `identity:get_project(project_id=60981a9be15c4b04b78b97afc4285aa0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:39.002753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:39.002971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 202/418] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:38 2022] GET /identity/v3/projects/60981a9be15c4b04b78b97afc4285aa0 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:39.008357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:39.041680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:39.130704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8278e28-e51a-46fa-81ca-67f05c9623ba None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:39.176313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:39.176313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:39.176313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:39.176655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:39.187093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a4bf1-d748-47f7-8460-6b8159228c02 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:39.199259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:39.200031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:39.200864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 217/419] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:39.213066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:39.273310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:39.273988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:39.274274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:39.274556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] PATH_INFO: `/v3/projects/60981a9be15c4b04b78b97afc4285aa0/groups/36530794718f437d8addf0f0efaea6a6/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:39.280653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] RBAC: Authorizing `identity:create_grant(project_id=60981a9be15c4b04b78b97afc4285aa0, group_id=36530794718f437d8addf0f0efaea6a6, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:39.282089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:39.322787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ec745-d320-4874-b812-7fe1b272b957 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:39.342902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:39.371195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:39.371195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:39.371195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:39.371195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:39.371612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:39.371612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:39.375186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:39.378534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 203/420] 10.222.0.62 () {58 vars in 1442 bytes} [Tue Sep 13 09:57:39 2022] PUT /identity/v3/projects/60981a9be15c4b04b78b97afc4285aa0/groups/36530794718f437d8addf0f0efaea6a6/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:57:39.381130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:39.414065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 218/421] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:39.415135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:39.522714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:39.629608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f4c04-4a29-4a52-9b29-21155c3ad445 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:39.642960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8c938-4c95-4973-8f29-644405d348e0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:39.693552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:39.695478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:39.695770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:39.696071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:39.731190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:39.751196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:39.752869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:39.753629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:39.754208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:39.754208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:39.754208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:39.754208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:39.758254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 204/422] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:39.759190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:39.872207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:39.892174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:39.893628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.007119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 219/423] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:40.009907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7476c9-7e9d-43c2-ac84-ce31c5dd547d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:40.012379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:40.043080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:40.090813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:40.091496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.091781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.092054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.092711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:40.093012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:40.094386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.120979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 205/424] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:40.127331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:40.160524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc694-0b1a-4e2e-9c34-9ed6168537d8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:40.163610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:40.205625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:40.206648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.207239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.208708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.210678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:40.212324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.217224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 220/425] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:40 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:40.221880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:40.313384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f5fd6-2026-442e-a7fc-a06c49a6d7d7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:40.362475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75005908-3f3e-4278-aa46-79e0953f32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:40.363404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75005908-3f3e-4278-aa46-79e0953f32db None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.363833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75005908-3f3e-4278-aa46-79e0953f32db None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.364250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75005908-3f3e-4278-aa46-79e0953f32db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.417839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b074a-8640-4ad0-b244-7d811a379d76 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:40.467576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:40.483851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75005908-3f3e-4278-aa46-79e0953f32db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:40.486219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75005908-3f3e-4278-aa46-79e0953f32db None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.489802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f1503-3e83-421d-9f14-4116828bad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:40.489973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f1503-3e83-421d-9f14-4116828bad47 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.490120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f1503-3e83-421d-9f14-4116828bad47 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.490394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f1503-3e83-421d-9f14-4116828bad47 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.492220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f1503-3e83-421d-9f14-4116828bad47 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:40.493663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f1503-3e83-421d-9f14-4116828bad47 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.499186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 221/426] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:40 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:40.500971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee97483-3df6-4683-b030-4d45b835f65a None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.501806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee97483-3df6-4683-b030-4d45b835f65a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.502093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee97483-3df6-4683-b030-4d45b835f65a None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.503566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 222/427] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:40.506051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee97483-3df6-4683-b030-4d45b835f65a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:40.522101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:40.604556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 206/428] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:40.605797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b82f9-f4a3-42b0-aeaa-fa615a1b88b1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.606079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b82f9-f4a3-42b0-aeaa-fa615a1b88b1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.606354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b82f9-f4a3-42b0-aeaa-fa615a1b88b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.618859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee97483-3df6-4683-b030-4d45b835f65a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:40.625176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d83b82f9-f4a3-42b0-aeaa-fa615a1b88b1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:40.626947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d83b82f9-f4a3-42b0-aeaa-fa615a1b88b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:40.656516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:40.657133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.657413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.657708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.658475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:40.658783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:40.660531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1883d3-119c-4850-b78f-40116a29554b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.673751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 223/429] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:40 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:40.673751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222a67-c753-41df-8ca3-bbc036ffe19c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.673751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222a67-c753-41df-8ca3-bbc036ffe19c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.673751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222a67-c753-41df-8ca3-bbc036ffe19c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.673751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 224/430] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:40.673751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef222a67-c753-41df-8ca3-bbc036ffe19c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:40.705808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 207/431] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:40.707085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.707578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.708018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.731084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:40.744047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:40.747078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:40.787554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:40.820047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef222a67-c753-41df-8ca3-bbc036ffe19c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:40.888185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 208/432] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:40.891074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:40.892009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:40.892438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:40.892857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] PATH_INFO: `/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:40.894240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] RBAC: Authorizing `identity:get_project(project_id=5a44c7b4a430429e94e14dfb7e2c81a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:40.895869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:40.897908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 225/433] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:40 2022] GET /identity/v3/projects/5a44c7b4a430429e94e14dfb7e2c81a5 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:40.905576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.007098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76c1d9-c136-48a7-a6fe-97be344514db None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.043880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.044556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.044837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.045104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.046189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:41.047838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:41.050338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 209/434] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:40 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:41.056087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.066064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.073459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.201069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cff74d-ec36-4803-b71b-7da1aeca23db None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.219042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93676a-b064-4897-b167-54e490ac08e6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.250286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.250900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.253996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.253996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.253996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:41.253996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:41.253996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:41.261174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 226/435] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=5a44c7b4a430429e94e14dfb7e2c81a5&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:41.261943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.263925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.264789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.265072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.266264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.278877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.285626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:41.293525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:41.294525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 09:57:41.295320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:41.299681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 210/436] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:41.301357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.408278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.422598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089ee1-6952-46b9-a39e-9958e0b3f7ed None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.464096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc889dec-a2ed-4e5a-9009-c518f8e31364 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.471480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.472126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.472406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.472679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.490834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:41.498363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:41.499265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:41.500055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 227/437] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:41.505969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.508136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.508748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.509066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.509340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.510006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:41.510310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:41.511774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:41.522718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 211/438] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:41.529413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.603784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.653108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536cfea-5873-41b3-add4-75df48daa7b0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.657928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a3c01-eb41-4501-9220-a5cecabf0d21 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.693302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.697032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.697976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.698538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.698995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.699871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:41.700370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:41.701886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac422b-bad5-4602-84c6-35f9e5b41031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.702053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:41.702494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac422b-bad5-4602-84c6-35f9e5b41031 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.702779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac422b-bad5-4602-84c6-35f9e5b41031 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.703054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac422b-bad5-4602-84c6-35f9e5b41031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.712857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 228/439] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:41.718787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.803503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac422b-bad5-4602-84c6-35f9e5b41031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:41.804875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac422b-bad5-4602-84c6-35f9e5b41031 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:41.858368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9aebc-d501-4196-b26a-0d21b33c8a34 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:41.887828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.893616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:41.894214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.894502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.911400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.911555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 212/440] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:41.911555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d525a-ce14-483a-b66c-bb0b14a25a87 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:41.911555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d525a-ce14-483a-b66c-bb0b14a25a87 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:41.911555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d525a-ce14-483a-b66c-bb0b14a25a87 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:41.911555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 213/441] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:41.911555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d525a-ce14-483a-b66c-bb0b14a25a87 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:41.934095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:41.961663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:41.963126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.007183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d525a-ce14-483a-b66c-bb0b14a25a87 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.040741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332eae6-9640-4e83-b332-7b7828df12fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.041385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332eae6-9640-4e83-b332-7b7828df12fe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.041712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332eae6-9640-4e83-b332-7b7828df12fe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.041980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332eae6-9640-4e83-b332-7b7828df12fe None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.043142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332eae6-9640-4e83-b332-7b7828df12fe None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.044576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332eae6-9640-4e83-b332-7b7828df12fe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.046210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 214/442] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:42.047340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.047626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.047897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.061182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 229/443] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 09:57:42.061182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.066799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:42.068030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:42.094766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:42.143866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 215/444] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:42.144707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.169503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:42.171025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723beec2-3bee-4c4c-8f9d-e73994e13282 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.210044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.210678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.210980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.211259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.212414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.213846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.215626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 230/445] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:42.221194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.248591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a5271-9c86-4d27-b144-09f33c458b34 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.285684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.286314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.286597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.286871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] PATH_INFO: `/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.288034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] RBAC: Authorizing `identity:get_project(project_id=4f84c0c22d344bdba56d9d3dccd426d0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.289550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.291639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 216/446] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/projects/4f84c0c22d344bdba56d9d3dccd426d0 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:42.296797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.320412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:42.323351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76474f45-eb4d-4610-94ae-74bfbc16911c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.362445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.363130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.363437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.363701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.364418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=28d0ae899af34488a7f267d899b9aff7, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:42.364779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.366257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.372205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 231/447] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=28d0ae899af34488a7f267d899b9aff7 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:42.404213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:42.413585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41354c91-bb6c-4eaf-b434-81007ae9f7af None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.415023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.453345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.454099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.456517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.456517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] PATH_INFO: `/v3/groups/b2f09a67e6fa4f54a813949e51f78473` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.456702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] RBAC: Authorizing `identity:get_group(group_id=b2f09a67e6fa4f54a813949e51f78473)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.458181 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.460191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 217/448] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/groups/b2f09a67e6fa4f54a813949e51f78473 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:57:42.465983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.536167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f665a-cdfe-4317-bb2a-dd58fa64433c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.576323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55ce12-cf74-4e0c-a188-bc9eeff83d49 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.580238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.580882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.581164 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.581440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.594886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:42.600672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:42.601579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:42.602385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 232/449] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:42.608653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.616219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.616825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.617102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.617369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.619662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] RBAC: Adding query filter params (group.id=b2f09a67e6fa4f54a813949e51f78473, role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:42.619662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.620499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.627682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 218/450] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/role_assignments?group.id=b2f09a67e6fa4f54a813949e51f78473&scope.project.id=4f84c0c22d344bdba56d9d3dccd426d0&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:42.639436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:42.730151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118659-5ab6-4e42-b5b3-82f6efcbf178 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.769548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.770166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.770445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.770714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.771414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:42.771718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.773119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:42.782712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 233/451] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:42.788680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:42.815985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:42.890730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336221ca-9a97-4fe4-9127-add9ae988364 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:42.926670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:42.927308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:42.927595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:42.927871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:42.994657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:42.996126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:43.019796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:43.091304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 219/452] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:43.096970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:43.179551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:43.193910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d4806-bb3a-4bcb-bf6a-3fcec131766e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:43.226302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:43.226905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:43.227204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:43.227476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] PATH_INFO: `/v3/projects/60981a9be15c4b04b78b97afc4285aa0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:43.228565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] RBAC: Authorizing `identity:get_project(project_id=60981a9be15c4b04b78b97afc4285aa0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:43.229983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:43.231599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 234/453] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:57:43 2022] GET /identity/v3/projects/60981a9be15c4b04b78b97afc4285aa0 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:43.237170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:43.333748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c9557-0577-4acc-82d2-0b41dfc37561 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:43.355417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:43.365999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68d715-f98f-4647-bb2f-7b117c273c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:43.366613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68d715-f98f-4647-bb2f-7b117c273c85 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:43.371288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68d715-f98f-4647-bb2f-7b117c273c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:43.371288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68d715-f98f-4647-bb2f-7b117c273c85 None admin] PATH_INFO: `/v3/groups/36530794718f437d8addf0f0efaea6a6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:43.371288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68d715-f98f-4647-bb2f-7b117c273c85 None admin] RBAC: Authorizing `identity:get_group(group_id=36530794718f437d8addf0f0efaea6a6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:43.371288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68d715-f98f-4647-bb2f-7b117c273c85 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:43.371288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 220/454] 10.222.0.62 () {56 vars in 1180 bytes} [Tue Sep 13 09:57:43 2022] GET /identity/v3/groups/36530794718f437d8addf0f0efaea6a6 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:43.377563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:43.478836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0458a-81b4-47b9-abe3-f7a312944f88 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:43.511543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:43.512161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:43.512440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:43.512707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:43.513420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] RBAC: Adding query filter params (group.id=36530794718f437d8addf0f0efaea6a6, role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=60981a9be15c4b04b78b97afc4285aa0) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:43.513726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:43.515131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70d1d3-51bf-4686-8299-35b66a8a31e7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:43.520707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 235/455] 10.222.0.62 () {56 vars in 1376 bytes} [Tue Sep 13 09:57:43 2022] GET /identity/v3/role_assignments?group.id=36530794718f437d8addf0f0efaea6a6&scope.project.id=60981a9be15c4b04b78b97afc4285aa0&role.id=435125566e954d20a3b8273b7c806b98 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:46.077684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f28e-3db4-4ee4-839b-45728cf3740e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.078201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f28e-3db4-4ee4-839b-45728cf3740e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.078276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f28e-3db4-4ee4-839b-45728cf3740e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.080487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 221/456] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:46.085993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea25ab-2cb0-4ede-b0a8-fe3a99c35cc5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.086719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea25ab-2cb0-4ede-b0a8-fe3a99c35cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.087302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea25ab-2cb0-4ede-b0a8-fe3a99c35cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.111656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d3ea25ab-2cb0-4ede-b0a8-fe3a99c35cc5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:46.114773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d3ea25ab-2cb0-4ede-b0a8-fe3a99c35cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:46.139569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:46.217719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 236/457] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:46.445209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dfb1e-4494-40c9-ba0a-4545d1e4a97b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.445553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dfb1e-4494-40c9-ba0a-4545d1e4a97b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.445761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dfb1e-4494-40c9-ba0a-4545d1e4a97b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.447495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 222/458] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:46.471199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.471199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.471199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.496115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:46.497593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:46.561648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e5af7-1dc2-49e6-8bf0-62da3d8cebce None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.561648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e5af7-1dc2-49e6-8bf0-62da3d8cebce None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.561648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e5af7-1dc2-49e6-8bf0-62da3d8cebce None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.561648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 223/459] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:46.567645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.568176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.568650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.585459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 237/460] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:46.587494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:46.588908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:46.635084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:46.667557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 224/461] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:46.791270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:46.819115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710e490-c032-4b61-a6a8-ceddca8d91ce None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:46.869048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:46.869713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:46.869998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:46.870276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:46.884564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:46.886046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:46.905264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:46.906419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:46.908032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:46.908032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:46.908032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:46.908032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 238/462] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:46.913268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:46.974782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.019476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4818a-493f-47c8-8b4c-a4ff4b540024 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.046721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:47.064220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.064882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.065280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.065552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.066201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:47.066505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.067909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.080546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 225/463] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:47.081290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.081745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5265c1-2332-454c-8e1a-1c5d641dcc69 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.116191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:47.121961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.125680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.125680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.125680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.134467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.136366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d0303a37-cf15-42d0-83f4-ffe7cd2b42e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:47.152260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:47.153318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:47.154337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:47.154337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:47.154337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:47.155938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 239/464] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:47.157889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d58f4-1d1d-4abd-996a-072ba1a0c90d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.158322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d58f4-1d1d-4abd-996a-072ba1a0c90d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.158723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d58f4-1d1d-4abd-996a-072ba1a0c90d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.160123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 240/465] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:47.160929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d58f4-1d1d-4abd-996a-072ba1a0c90d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.190430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1d521-54af-4d17-b780-4b7826859b33 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.227021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.227620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.227900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.228172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.239582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.241143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:47.256793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:47.257707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:47.258579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:47.258579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:47.258579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:47.258579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 226/466] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:47.258796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.270025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d58f4-1d1d-4abd-996a-072ba1a0c90d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.295118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:47.304112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3da53-d70b-4cc3-be96-a90d9869f2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.304721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3da53-d70b-4cc3-be96-a90d9869f2f9 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.308673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3da53-d70b-4cc3-be96-a90d9869f2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.308673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3da53-d70b-4cc3-be96-a90d9869f2f9 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.308673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3da53-d70b-4cc3-be96-a90d9869f2f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.308673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3da53-d70b-4cc3-be96-a90d9869f2f9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.368692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3f671-5052-4144-ba48-9251410c59dc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.405823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.406718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.407006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.407903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.408548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:47.408857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.410225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.415809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 241/467] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:47 2022] POST /identity/v3/users => generated 246 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:47.417201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.420019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.420019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.442352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 227/468] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:47.442352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.448278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:47.449864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:47.485872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:47.538478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 242/469] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:47.539707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.555329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:47.562809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96098d-acae-448f-9ed8-41f047af6860 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.610155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.610846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.611124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.611661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.612369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:47.612682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.614064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.637684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 228/470] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:47.643278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.706384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ab53-7357-40d6-bb4a-2baa8d962dc3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.749625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.749920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.750220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.750599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.751831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.753931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.776110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fefe6-0843-4370-89c2-fb924e81f99d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:47.817185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1f61b-9897-4700-a508-032f868a9db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:47.817841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1f61b-9897-4700-a508-032f868a9db9 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.818143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1f61b-9897-4700-a508-032f868a9db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.818390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1f61b-9897-4700-a508-032f868a9db9 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.819028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1f61b-9897-4700-a508-032f868a9db9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:47.820621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1f61b-9897-4700-a508-032f868a9db9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:47.836720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 243/471] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:47 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:47.888082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:47.913448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 229/472] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:47 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:47.914865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540514b0-fc61-4bbf-b8c5-c7d4567455b1 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.923298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540514b0-fc61-4bbf-b8c5-c7d4567455b1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.923298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540514b0-fc61-4bbf-b8c5-c7d4567455b1 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.923298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 230/473] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:47.924255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:47.931979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:47.932303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:47.932585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:47.972725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:47.973997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:47.999617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:48.086854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dce541-681b-4ccf-9baa-e3f9e8dcd29f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:48.091762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 231/474] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:48.125440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:48.126699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:48.127026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:48.127352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:48.140814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:48.142376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:48.157858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:48.158786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:48.159686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:48.159686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:48.159686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:48.159686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 244/475] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:48.166529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:48.284581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134446a-10d8-40ab-9a2e-b5995c4f069b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:48.320229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:48.320844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:48.321126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:48.321401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:48.322049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:48.322356 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:48.323761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:48.332550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:48.336174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 232/476] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:48.345237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:48.388814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:48.486145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:48.521228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a990c5-864c-426b-a863-3ed9ddb73101 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:48.523869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb2087-686d-4583-9230-39005cfc97ac None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:48.562976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:48.563754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:48.564032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:48.564304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:48.564985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:48.566336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:48.592770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:48.593378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:48.605023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:48.605023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:48.605023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:48.605023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:48.625866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.sql.core [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 09:57:48.625866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 09:57:48.625866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [parameters: {'id': 'bcf30eb4746a40ab8fd86715631b36bc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 09:57:48.625866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81737) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application conn.query(q) Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result.read() Sep 13 09:57:48.643576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 09:57:48.644539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application next(self.gen) Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application yield resource Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application next(self.gen) Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise self.value Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application session.commit() Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 09:57:48.645430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.session.flush() Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application compat.raise_( Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 09:57:48.646545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 09:57:48.647743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application conn.query(q) Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result.read() Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 09:57:48.651608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [parameters: {'id': 'bcf30eb4746a40ab8fd86715631b36bc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:48.652595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 09:57:48.653732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:48.654743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 233/477] 10.222.0.62 () {60 vars in 1131 bytes} [Tue Sep 13 09:57:48 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:48.654743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:48.690846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 245/478] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:48 2022] POST /identity/v3/users => generated 249 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:48.736532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:48.790019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3cbe1-03c0-4ecb-9620-c43b3f067074 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:48.842732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944393ef-714d-43a4-9189-aeca69765282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:48.843429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944393ef-714d-43a4-9189-aeca69765282 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:48.843710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944393ef-714d-43a4-9189-aeca69765282 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:48.843983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944393ef-714d-43a4-9189-aeca69765282 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:48.856176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944393ef-714d-43a4-9189-aeca69765282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-944393ef-714d-43a4-9189-aeca69765282 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:48.869940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:48.870933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:48.871759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 234/479] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:57:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:48.877329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:49.005139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c0d05-7e28-4910-997e-e3952311baec None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:49.042583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:49.049355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:49.049985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:49.050266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:49.050627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:49.051376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:49.051683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:49.060101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ef02e-c05a-44a8-a5bd-3d32a012b35f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:49.065381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 246/480] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:50.262786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922052-2e31-434c-b2e9-ad81cd4d0577 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.263653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922052-2e31-434c-b2e9-ad81cd4d0577 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.264118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922052-2e31-434c-b2e9-ad81cd4d0577 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.265891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 235/481] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:50.271385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363e07-d308-47b9-9adf-39707c72fcd7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.272123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363e07-d308-47b9-9adf-39707c72fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.272539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363e07-d308-47b9-9adf-39707c72fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.294621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ff363e07-d308-47b9-9adf-39707c72fcd7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:50.296071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ff363e07-d308-47b9-9adf-39707c72fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:50.403414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 247/482] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:50.566359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90ac9-0bbb-4684-8dac-f7ea562181c7 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.566679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90ac9-0bbb-4684-8dac-f7ea562181c7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.566873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90ac9-0bbb-4684-8dac-f7ea562181c7 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.568461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 236/483] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:50.573982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.574271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.574543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.600228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:50.601469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:50.615745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853903ba-42e1-4c25-a49f-fde2ad1f6844 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.616044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853903ba-42e1-4c25-a49f-fde2ad1f6844 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.616296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853903ba-42e1-4c25-a49f-fde2ad1f6844 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.617798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 237/484] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:50.628290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.628575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.628843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.659257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:50.660445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:50.715020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 248/485] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:50.754669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 238/486] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:50.760914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:50.823267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:50.883729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f9f0a-8dd0-4d6c-b5d5-69e54de26665 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:50.927989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:50.928629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:50.928911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:50.929185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:50.947541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:50.952406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:50.953270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:50.954098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 249/487] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:50.959457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.046052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.057774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.103700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b716f88-84b1-477d-9e32-bab3ff1a12c9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.168108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.169076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.169638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.170775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.171835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:51.173331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.174980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.185788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 239/488] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:51.186883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.229993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081c6f8-8854-4b94-8642-4711407f34b0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.245257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.266321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.267614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.267897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.268226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.279180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:51.286304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:51.287335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:51.288179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 250/489] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:51.288820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.330293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee20e1-7ce4-496b-8f50-9851e3b413d2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.349087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.378661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.379649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.380102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.380617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.405305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:51.411959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:51.412902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:51.413662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:51.418339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 240/490] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:51.419308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.426114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f853-f59d-4c1d-8c55-00816433b7a8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.467716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.468814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.469346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.470006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.483135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.484968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:51.500450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:51.501930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:51.502738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:51.502738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:51.502738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:51.504555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 251/491] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:51.505529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.527691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd003f-b6a8-4f5d-9a6d-aef121c4606e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.535388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.564690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.565619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.565903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.567362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.567362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:51.567362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.568760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.578391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 241/492] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:51.580133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.617293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddc851-cace-42b2-b181-15e053aee733 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.621747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.654006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.654624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.654972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.655281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.655946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] RBAC: Adding query filter params (name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:51.656248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.657668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde91bd9-76eb-4e57-b408-c04b071f336e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.670874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 252/493] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:51.670874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33511e80-cacb-48eb-95b6-def3ce958b81 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.670874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33511e80-cacb-48eb-95b6-def3ce958b81 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.670874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33511e80-cacb-48eb-95b6-def3ce958b81 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.670874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 253/494] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:51.670874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33511e80-cacb-48eb-95b6-def3ce958b81 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.693147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa021d8-4b00-4a6f-bb61-9bf4e27db7c8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.734049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.734660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.734931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.735758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.736682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:51.737394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.739012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.756985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 242/495] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:51.756985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.765729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.778571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33511e80-cacb-48eb-95b6-def3ce958b81 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.816682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.817419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.817694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.817965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.829039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.830464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-2dcdb834-104c-4f4e-9a80-18dcbeca5109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:51.845867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:51.846776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:51.847646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:51.847646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:51.847646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:51.847646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 254/496] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:51.848354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.848529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.848782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.863993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:51.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6747c6-adb0-403a-a2f0-bf03c36fdae2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:51.868224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:51.869485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:51.904558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:51.905124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:51.905405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:51.905834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:51.917534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:51.919011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:51.939288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:51.940230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:51.946090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:51.946090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:51.946090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:51.946090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 243/497] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:51.946090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:51.951128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 255/498] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:51.959611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.030191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.061206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ebac1-08d8-4d2f-b92b-bb247fc5e72f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.065401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266df0d-7fc4-4f1a-9e4d-429e1bd76131 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.100821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.101460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.101742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.102019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.111350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.113235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.114668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.115564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.116137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.116406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.117260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:52.117571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.118929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502c53f-7abe-4361-b2d9-6c6b7655aa41 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:57:52.126159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:52.127059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 244/499] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:52.132975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.146235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 256/500] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:52.146235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acddf0b-f88b-4a74-9348-5f7ed786cfd0 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.146235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acddf0b-f88b-4a74-9348-5f7ed786cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.146235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acddf0b-f88b-4a74-9348-5f7ed786cfd0 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.146235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 257/501] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:52.146235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acddf0b-f88b-4a74-9348-5f7ed786cfd0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.151126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.242159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eba25-496c-4148-8df1-5e091cfc79fa None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.253853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acddf0b-f88b-4a74-9348-5f7ed786cfd0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.274435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.275035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.275345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.275618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.276250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:52.276551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.277919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.283457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.289067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.289716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.289977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.290251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.290739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 245/502] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:52.291255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:52.291561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.297371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.297502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ee11-782c-4549-ad45-a80ffdcfc8dd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.330052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 258/503] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:52.331416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.332428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.332428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.350375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:52.351617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:52.361755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.392204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187e9f-c038-4fce-9790-c3bb0431379e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.423964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 259/504] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:52.435410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.441116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.441743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.442021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.442288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.450152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.451623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:57:52.463670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:52.464559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 246/505] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:52.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.535886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e8b9-2c27-4f8e-ab9b-1ec425ac524a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.555054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.571360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.582279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.582279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.582279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.582522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fcd00-d489-4b38-90b2-b3763551796d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.583401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:52.588231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:52.589287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:52.590280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:52.594191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 260/506] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:52.596108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.612784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.618607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.621796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.621796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.621796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.627380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.628816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:57:52.640415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:52.641802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:52.642703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 247/507] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:52.643280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.705860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e42540-8545-4826-94c5-9646c0a64995 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.742662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.743307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.743594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.743866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.749832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7d2f4-7757-46c1-8921-c0bc16c35b56 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.783809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.784410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.784692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.784967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.785692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:52.786053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.787444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.790655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.812067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.814050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.823806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 248/508] 10.222.0.62 () {56 vars in 1137 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:52.824625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.873830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:52.912979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 261/509] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:52.913809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.927183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb361896-2d43-4736-b204-b96a94cb9e64 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:52.960503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:52.961102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:52.961383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:52.961653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:52.962310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:52.962612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:52.964054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:52.971711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 249/510] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:52.972492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:52.988692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.009741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb04f-9ac3-4a1d-b544-5a342e5a771d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.044470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.044998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.045275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.045542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.046251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:53.046549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.048010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.070574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 262/511] 10.222.0.62 () {56 vars in 1137 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:53.071425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.077506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366b55-5ab1-4c6b-8391-6db31844f80d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.089356 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.112345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.112963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.113240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.113510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.123474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.124910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:53.139463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:53.140390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:53.141383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:53.141383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.141383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:53.141383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 250/512] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:53.141949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.172422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09553b7-325b-44ca-b6ec-da544a64eb1b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.206341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.206941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.207334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.207620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.211016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.239715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f44b0-ee7b-4b9d-8230-77475a04beee None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.271784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.272403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.272685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.272956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.273285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.274741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.283107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.284563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:53.298984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:53.299914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:53.300738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:53.300738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.300738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:53.300738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 251/513] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:53.300959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.316616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.377336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 263/514] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:53.378184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.400492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300164b5-520b-480f-a8aa-534a28f021b5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.413758 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.434715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.435374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.435659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.435935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.446607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.448088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:53.462307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:53.463249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:53.464108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:53.464108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.464108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:53.464108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 252/515] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:53.464317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.484702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821021b2-21ef-4ffd-b598-c0c9e7e5de79 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.518492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.519094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.519406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.519702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.532838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.569308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d488786-6af9-4f87-91c4-2a48f0ab2ad9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.587384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.588887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.601804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.602405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.602685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.602959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.603636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:53.603942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.605300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.617121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 253/516] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:53.617899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.622827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.687020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 264/517] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:53.687889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.720954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76feda1-3798-41cf-ad31-508b3cbf32c3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.746392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.754583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.755185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.755482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.755752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.756378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:53.756679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.758017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.769392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 254/518] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:53.770170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.786285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae67ee-0733-4c17-8ebe-ab59a374abe5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.820852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.821411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.821689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.821962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.824568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.832567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.834007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:53.848961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:53.849895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:53.850925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:53.850925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:53.850925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:53.850925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 265/519] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:53.851272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.872380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e91de-500c-42dd-ab60-e6a3828cc426 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.905252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.905859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.906141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.906415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.907616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:53.907923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.909279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:53.921691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 255/520] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:53.922474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:53.948090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453fee0-1a02-4e12-9f0e-46cd27f9258c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:53.967773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:53.981630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:53.982239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:53.982515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:53.982788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:53.983459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:53.984916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.019802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce240b6-a6e4-40e0-8c26-96b9bafb76ce None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.043705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 266/521] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:57:53 2022] POST /identity/v3/users => generated 247 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:57:54.044516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.054031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.054689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.054973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.055279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.066870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.068415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:54.085289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:54.086611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:54.087658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:54.087658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:54.087658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:54.087658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 256/522] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:54.088606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.123961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.185614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c530e-f229-41ef-a056-0d624db2adf9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.209614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.213918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a77317-7571-4d32-86c3-fca3aedf6efc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.226282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.226929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.227251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.227606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.236150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.237617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:57:54.249482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:54.250370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:54.251908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 267/523] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:54.252909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.259116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.260005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.260283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.260558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.261212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:54.261516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.262886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.276685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 257/524] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:54.277539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.366242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f0b70-fa6f-4a63-a396-a0dbdcb53374 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.369854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.384547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef04a91-ece3-44f9-91b8-2bd540d88b99 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.408022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.408659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.408944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.409210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.418730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.420216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.423495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.424177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.424523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.424861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.425571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:54.425943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.427497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:57:54.433278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:54.434408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 09:57:54.435523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:54.439467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 268/525] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:54.440450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.443251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 258/526] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:54.443251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.465749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.542609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6a38-8ab3-41f8-8af2-743de3c83f0a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.550117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37bb44-b050-4f78-b126-7716ecd700b5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.576988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.578173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.578463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.578761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.580022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:54.580332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.581794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.589122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.589757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.590035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.590305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.597242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.598663 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.604969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 269/527] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:54.604969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:57:54.616518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:54.617372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:54.618220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 259/528] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:54.618686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.637556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.699851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.719086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d88c33-1d23-437d-bf7e-ba5abce2bc9c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.722137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e048d-e0fe-4e82-b2b5-2dd2d9c764b5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.758187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.758820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.759102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.759415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.760524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.761124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.761432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.761676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.762395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:54.762699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.764171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.766345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.767819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:57:54.779297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:54.780312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 260/529] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:54.781125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.784854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 270/530] 10.222.0.62 () {56 vars in 1139 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:54.785865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.873359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:54.892554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35a84f-438f-4b55-87f6-29dfa2f1bde6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.895925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0112a-7cf3-4893-92ba-f747678d6518 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:54.929239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.930182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.930290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:54.930785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.930890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:54.931121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:54.931427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.931807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:54.932157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=81eb5931848749a1ad22d9d228899cda) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:54.932470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.933098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:54.933600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:54.933901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.935456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:54.939974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 261/531] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/role_assignments?user.id=81eb5931848749a1ad22d9d228899cda&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:54.940862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.948859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 271/532] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:54.950048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:54.951537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:55.076997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b43ee-80dd-46ee-83fa-7f65ef9fa475 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:55.095896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25453138-9324-4102-b5fd-9dea763e3426 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:55.124324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:55.125175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:55.125641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:55.126055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:55.126990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:55.127523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:55.129596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:55.143203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:55.144474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:55.144474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:55.144474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:55.156558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 262/533] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:55.158008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:55.158879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:55.220560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:55.222036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:55.228010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:55.276383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4d52a-02b5-4a2e-964b-34afd6cde110 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:55.317815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:55.318992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:55.319312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:55.319596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:55.320337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=5950076b981b43feba3185f1fb513729) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:55.320650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:55.322555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:55.330298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 263/534] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:55 2022] GET /identity/v3/role_assignments?user.id=5950076b981b43feba3185f1fb513729&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:55.331482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:55.354778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 272/535] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:55.367198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:55.465412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:55.554633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e8d16-1ade-4ecd-b3b0-28fa8fbaca9d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:55.559686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28a4ae-8d54-4665-be13-3dd22ecb7342 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:55.597853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:55.598492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:55.598773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:55.599049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:55.610439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:55.612608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:55.614514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:55.632498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:55.633437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:55.634299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:55.634299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:55.634299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:55.638193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 273/536] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:55.639400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:55.642757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:55.643650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:55.644100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:55.644378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:55.645114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=566f088827ab4710bf54d64415831a1c) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:55.645426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:55.646872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:55.657911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 264/537] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:55 2022] GET /identity/v3/role_assignments?user.id=566f088827ab4710bf54d64415831a1c&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:55.759851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:55.769011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e32d18-6e25-4c8f-a0d4-626b77435708 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:55.807824 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:55.808853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:55.809356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:55.809838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:55.810728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:55.811277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:55.812975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:55.830549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 274/538] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:55.835971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:55.963156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b23f95-e37b-4714-897e-d2497ef85f87 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:56.021316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:56.021979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:56.022263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:56.022538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:56.033303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:56.034796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:57:56.056671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:56.057570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:56.058808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 265/539] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:56.065994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:56.183400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d3e5b-24d7-4efe-8d23-cfcd12a0c77c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:56.219641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:56.220588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:56.221094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:56.221568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:56.222454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:56.222958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:56.224614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b414f86-fa6d-40a5-a3f1-4aba1424f4d0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:56.242092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 275/540] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:57:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:56.244066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:56.298477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:56.370965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a1bcd-3599-4c2c-bea3-1807e9603d34 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:56.418622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:56.419286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:56.419571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:56.419852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:56.420611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=5889d3606c904dac9b4674efed1f94a6) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:56.420923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:56.422944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b73292-02b3-4d35-bac0-a805dd329c13 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:56.434238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 266/541] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:57:56 2022] GET /identity/v3/role_assignments?user.id=5889d3606c904dac9b4674efed1f94a6&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:56.622038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10891ef8-2cdf-4718-899c-0587ef1269dc None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:56.622564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10891ef8-2cdf-4718-899c-0587ef1269dc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:56.622638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10891ef8-2cdf-4718-899c-0587ef1269dc None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:56.624414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 276/542] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:56.630159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:56.630658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:56.630731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:56.651629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:56.652827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:56.730653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 267/543] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:57.191428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10891ef8-2cdf-4718-899c-0587ef1269dc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:57.208882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:57.309605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10891ef8-2cdf-4718-899c-0587ef1269dc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:57.345064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:57.345707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:57.345990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:57.346266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:57.358064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:57.363622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:57.364480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:57.365268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 277/544] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:57.372494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:57.444087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:57.515375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0fbab-e36e-4e09-9cf5-b7d5c6921f4e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:57.554400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:57.555050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:57.555598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:57.555877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:57.556569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:57.556881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:57.558304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d0ff8-fe99-469d-af06-087612623f77 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:57.568125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 268/545] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:57.573818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:57.676410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa0da-75d8-4f1e-9fac-17b3744b5b0d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:57.677316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa0da-75d8-4f1e-9fac-17b3744b5b0d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:57.677766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa0da-75d8-4f1e-9fac-17b3744b5b0d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:57.679691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 269/546] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:57.684825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ec14a-8102-467d-9a53-43a6a968041d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:57.685314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ec14a-8102-467d-9a53-43a6a968041d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:57.685795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ec14a-8102-467d-9a53-43a6a968041d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:57.691529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:57.713413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5e5ec14a-8102-467d-9a53-43a6a968041d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:57.714949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5e5ec14a-8102-467d-9a53-43a6a968041d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:57.720344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac418ea-234a-4e87-88b9-fab391bbe124 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:57.760549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:57.761443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:57.761913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:57.762329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:57.785850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:57.787737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:57.814542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:57.815735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:57.817142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:57.817142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:57.817142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:57.819276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 278/547] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:57.824691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:57.858251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 270/548] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:57.979018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126a4b5-9885-4a5c-b4dc-babcbd923195 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.023190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.026882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.027828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.028260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.036367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.036367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:58.036367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:58.036367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:58.045382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba23f-97a7-43b2-a6f9-1a21778d8526 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.045382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba23f-97a7-43b2-a6f9-1a21778d8526 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.045382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba23f-97a7-43b2-a6f9-1a21778d8526 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.045382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 271/549] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:57:58.049232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 279/550] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:58.050576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.050875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.051150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.054904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.076770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:58.077989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:58.137389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.224548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 272/551] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:57:58.225507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.231741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e02685-4b96-46ed-97ed-a9801d65423d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.275799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.276438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.276727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.277017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.285221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:58.286866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0ab2e96e-89e2-48c7-ab09-1a502ecd3925 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:57:58.304943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:58.305882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 280/552] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79fe19-cb70-4e58-8110-2bd6ca3b40cb None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.306784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79fe19-cb70-4e58-8110-2bd6ca3b40cb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.307667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79fe19-cb70-4e58-8110-2bd6ca3b40cb None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.307667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 281/553] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:58.307667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79fe19-cb70-4e58-8110-2bd6ca3b40cb None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.338500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6c56d-d579-4da4-a521-fae60860024a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.372078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.375332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d6baa-8547-4ec1-b19f-5e215c8ae91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.376275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6baa-8547-4ec1-b19f-5e215c8ae91b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.376770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6baa-8547-4ec1-b19f-5e215c8ae91b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.377296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6baa-8547-4ec1-b19f-5e215c8ae91b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.397936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d6baa-8547-4ec1-b19f-5e215c8ae91b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a54d6baa-8547-4ec1-b19f-5e215c8ae91b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:58.399723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:58.400798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:58.401813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:58.408770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 273/554] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:58.410110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.410496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.410890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.429387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:58.430664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:58.455351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.460689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79fe19-cb70-4e58-8110-2bd6ca3b40cb None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.504423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 274/555] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:58.511218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.514294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.514965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.515277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.515555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.516913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:58.517222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:58.518672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:58.543367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 282/556] 10.222.0.62 () {56 vars in 1135 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:58.547791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.620867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee2b8-c97f-4463-929f-665505a390f6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.664321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.665023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.665641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.665985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.666716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:58.667089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:58.668577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:58.679203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 275/557] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:58.679203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.682511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b775d-81fa-4db2-a992-bd538f0f09f0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.684936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.705078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.720200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.720912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.721264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.722667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.733675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:58.738680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:58.739684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:58.740509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:58.744658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 283/558] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:58.745529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.801581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d4bbc-9fb2-4526-a19d-d9794d65b484 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.837299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.837913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.838194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.838711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.860181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ebc6a-9ed9-47c4-9f11-2b45c67a2ba0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:58.900919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:58.901709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:58.901942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:58.902215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:58.908737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:58.910186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:58.913620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:58.915098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:58.928611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:58.931605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:58.932684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:58.933777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:58.933777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:58.933777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:58.937325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 284/559] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:58.938750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:58.958590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.020791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 276/560] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:57:59.021922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.052086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7b368-b570-4527-acc2-a0857e4dc808 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.087865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.088470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.088748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.089016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.089669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] RBAC: Adding query filter params (name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:59.089967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.091405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.099454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 285/561] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:59.100312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.126856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afad20-8407-431e-a04a-b2bbd8d74eb6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.143692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.160572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.161205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.161485 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.161754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.174124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:57:59.178772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:57:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:59.180089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:59.180913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 277/562] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:59.181606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.202537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38d974-4fee-48e3-b014-14ecc70ce760 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.237223 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.237841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.238156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.238435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.239137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:59.239468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.240846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a9b1c-6be4-4f94-9d52-9de95a9d9193 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.252864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 286/563] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:59.252864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97019b-312c-4c55-8a1a-42bba82a3dfa None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.252864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97019b-312c-4c55-8a1a-42bba82a3dfa None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.252864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97019b-312c-4c55-8a1a-42bba82a3dfa None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.252864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 287/564] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:57:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:57:59.252864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97019b-312c-4c55-8a1a-42bba82a3dfa None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.280013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8857-dd1a-4f92-90ea-033ba0216270 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.313858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.314406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.314680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.314952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.324966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.326395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:59.339952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:59.340912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:59.341812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:59.341812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:59.341812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:59.341812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 278/565] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:59.341812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.362068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97019b-312c-4c55-8a1a-42bba82a3dfa None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.370342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.398965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.399622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.399937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.400207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.406655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.410412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.411867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:59.426937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:59.427986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:59.428854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:59.428854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:59.428854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:59.428854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 288/566] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:59.429074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.445533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435b12d-8c3b-48db-bbbe-ba0ec7693c80 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.478994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.479634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.479918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.480190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.480848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:59.481155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.482511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca932a-cfd0-4cd2-bae3-892bd2b55664 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.490822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 279/567] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:59.491985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.492270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.492544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.510076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:57:59.511331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:57:59.526445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362d4d0-93ad-4fd6-a6c2-eb3189f0a72c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.560382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.561031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.561311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.561577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.571502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.572881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.581071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 280/568] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:57:59.581907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:57:59.584379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:59.585285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:57:59.586191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:57:59.590029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 289/569] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:59.590836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.607954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.653993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.686681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ac7e4-3f5c-4239-9bae-1b7986352aac None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.708856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfadad-9dba-4738-847d-bb6828585a82 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.723474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.724089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.724370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.724643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.725283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:59.725586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.726941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.744016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 281/570] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:59.744016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.755078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.755715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.756006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.756273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.756911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:59.757215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.758571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.770752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 290/571] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:59.771588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.839101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.857527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b36790-c9f7-4bf8-bc33-5adde5fa770b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.876506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b0ce2-b10a-4dd6-be4f-3f521daecb29 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:57:59.886753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:57:59.890575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.891761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.892082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.894642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.902293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.903767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.910799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:57:59.911469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:57:59.911743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:57:59.912011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:57:59.912732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:57:59.913032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:57:59.914458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:57:59.919006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:57:59.920048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:57:59.921504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:57:59.921504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:57:59.921504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:57:59.921504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 282/572] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:57:59.922712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:57:59.931720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 291/573] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:57:59.932521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.022685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede504f-bbc7-4f76-adc7-e725d827c916 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.028761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a53f-856f-407f-b0a3-670154b5d6fd None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.055927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.056548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.056828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.057100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.058941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.064090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.064226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.066256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.066256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.066256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.066487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.072388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.073799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:00.076769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:00.077737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 283/574] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:00.083896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:00.086012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:00.087053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 292/575] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:57:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:00.092599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.118946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.181119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fd1ec-1ce1-4419-8401-18871079b7e6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.190356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4870c-f5b8-42bb-9f74-11a4da8ca43d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.214230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.214853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.215133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.215438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.223545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.225109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.225394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.225489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:00.225668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.226300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:00.226605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.227999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:00.230745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:00.231854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 284/576] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:00.232919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.247543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 293/577] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:00.247543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.290867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.331136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.335439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ff685-f65d-42f3-8009-9c6eb5f69d17 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.342669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d38cb-3085-4728-a3db-1434febc91e1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.368202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.368828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.369110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.369383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.376509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.377110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.377392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.377667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.378352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:00.378658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.380128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.380269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.381728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.394069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 294/578] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:00.394891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:00.397164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:00.398090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:00.398971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:00.398971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:00.398971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:00.398971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 285/579] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:00.399739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.494195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e703d05-4eaa-4094-ad43-7be3d1e580c6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.497953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c521a-25ce-458a-9a4a-bb7252d0c9a0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.505527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.528952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.529553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.529833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.530100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.530916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:00.531261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.532705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.533313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.533910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.534197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.534464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.535114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:00.535512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.536858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.545625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 286/580] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:00.546407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.553706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 295/581] 10.222.0.62 () {56 vars in 1137 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:00.554509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.562616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.646534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118454b-a1d3-427c-a570-50319122d238 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.652447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600ca22-0770-4d37-aa76-80ec4192f3fd None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.679817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.680518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.680717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.680995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.685948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.686540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.686818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.687088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.687849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=174cca9647954ad1afdd4e6b8fba18f4) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:00.688697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.688842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.690128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.690257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.695538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 296/582] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/role_assignments?user.id=174cca9647954ad1afdd4e6b8fba18f4&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:00.696312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:00.707024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:00.708241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:00.709151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 287/583] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:00.709721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.744050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.792379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.824954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474790c5-77fb-4a20-b9ba-a0db8251541e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.825932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1f187-3211-4dbb-bb36-c5e2985bf0b8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.861639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.862275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.862665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.862946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.865299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:00.865909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:00.866189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:00.866462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:00.867110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:00.867440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.868805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.874659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:00.876787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:00.881896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 297/584] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:00.882728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:00.891770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:00.892695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:00.893548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:00.893548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:00.893548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:00.893548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 288/585] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:00.893787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:00.979654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:00.989725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6fa8d-ba04-4686-bff3-7ec210cf0d7f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:00.998968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cd10b-ec0e-466f-ba10-c5dc8ce134c0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.017991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.025107 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.025741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.026067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.026337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.034918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.035773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.036050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.036374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.039223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:01.039314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.039405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.039478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.039546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:01.054809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:01.055982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:01.062156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:01.062156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:01.062156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:01.062156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 298/586] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:01.062156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.065647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 289/587] 10.222.0.62 () {56 vars in 1137 bytes} [Tue Sep 13 09:58:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:01.070229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.171041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541835a5-d9e3-4b94-b964-52e9a2fe794e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.175785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0f1e1-c2bb-429f-bf2e-fa91f34f3afc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.207155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.207982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.208112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.208364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.213092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.213696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.213973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.214252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.214871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:01.215191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.215349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.216573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.216743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.224555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:01.228532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:01.229386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:01.240140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:01.240578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 290/588] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:01.240656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 299/589] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:01.240724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.240798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.285172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.341066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605e11-f698-4339-bad6-ead82c25a8b8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.342328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a7451-a5dd-424e-8fca-8c1b13a9bafc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.377427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.378063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.378345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.378442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.378613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.379021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.379338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.379611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.380243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:01.380545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.382329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.392219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.393718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.400309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 291/590] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:01.405483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:01.409089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:01.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:01.410890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:01.410890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:01.410890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:01.414544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 300/591] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:01.415641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.470028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.513658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a0b0b-21ac-47a0-929a-fe422591c97b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.514231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.521165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0d2a-0de5-4914-9ab1-44cbe6223dab None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.546298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.546932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.547788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.548064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.560632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.561702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.562104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.563621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.564093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.564543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.565604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:01.566186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.567749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:01.576086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:01.577002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 09:58:01.577905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:01.579521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 292/592] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:01.580892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.587118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 301/593] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:01.588224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.689571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92c5d7-1279-4b5e-aea4-d426377fe521 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.690144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a775326-9d8e-432a-8053-c233768cecfc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.708352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.724073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.724674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.725016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.725405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.725990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:01.726294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.727110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.728332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.728422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.728422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.728422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.735836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.737406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.743822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 302/594] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:01.743822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:01.755985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:01.756898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:01.757763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 293/595] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:01.758754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.762421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:01.846797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ad734-49d8-456a-bef9-7d8b20e7a21a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.862933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a94be-16e5-4557-930c-ef55be4ed2ab None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:01.882109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.886144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.886434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.886714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.887470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:01.887780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.889206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.898327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:01.898989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:01.899298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:01.899572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/81eb5931848749a1ad22d9d228899cda/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:01.909271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 303/596] 10.222.0.62 () {56 vars in 1139 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:01.910298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:01.920153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=81eb5931848749a1ad22d9d228899cda, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:01.921602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:01.982628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:02.004017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 294/597] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:01 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/81eb5931848749a1ad22d9d228899cda/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:02.005125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.023990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1b54-2e5f-4cb6-b6f9-cf525da152fa None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.041759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:02.067703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.068757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.069338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.069854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.079463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.081325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:02.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:02.095620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:02.096480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:02.103720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 304/598] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:02.105129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.126633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e36e5-c79d-492b-b6e9-686b017ea2a5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.167866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.168687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.168798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.169034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.169731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:02.170040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.171518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.186015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 295/599] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:02.186873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.223722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479a433-84d1-498e-8c74-02059a9dedb7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.257809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:02.263988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.264838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.283495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.283495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.283495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.283495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:02.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:02.295714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:02.296602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:02.296602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:02.296602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:02.296602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 305/600] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:02.297262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.298079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:02.317034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0561b-05e3-4287-88d8-ce4acb9bd9a7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.365678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.366344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.366703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.366989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.368176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:02.368502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.370386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.385749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 296/601] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:02.386618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.422696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1a92-6b6a-4779-b3ab-d68f1c6a9146 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.467577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.469122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.469621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.470351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/5950076b981b43feba3185f1fb513729/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.489553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=5950076b981b43feba3185f1fb513729, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.491377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.505245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaf20f-8fe0-48b3-bff4-3915771d748c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.541536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.542221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.542505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.542770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.543467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:02.543772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.545136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.561438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 297/602] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:02.561438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.571315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:02.577875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 306/603] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:02 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/5950076b981b43feba3185f1fb513729/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:02.579639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.626356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:02.729514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba989e-22bd-48be-ad76-09b0ac7a2705 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.792699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e4bbb-893d-45de-be1d-26e3fd1639d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.793737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e4bbb-893d-45de-be1d-26e3fd1639d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.794216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e4bbb-893d-45de-be1d-26e3fd1639d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.794710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e4bbb-893d-45de-be1d-26e3fd1639d2 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/566f088827ab4710bf54d64415831a1c/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.794815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dc7e8-25a4-4781-b59b-1f7a8d26fc17 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:02.817107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e4bbb-893d-45de-be1d-26e3fd1639d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=566f088827ab4710bf54d64415831a1c, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.818505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e4bbb-893d-45de-be1d-26e3fd1639d2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.848417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:02.849084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.849344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.849612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.864037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:02.865489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:02.884606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:02.885474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:02.886350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:02.888991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 307/604] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:02.894127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:02.935366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 298/605] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:02 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/566f088827ab4710bf54d64415831a1c/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:02.938929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba113d0-d49a-4dee-9448-355ad3aa6192 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.939604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba113d0-d49a-4dee-9448-355ad3aa6192 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.940406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba113d0-d49a-4dee-9448-355ad3aa6192 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.943471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 299/606] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:02.955092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:02.955796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:02.956629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:02.981435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:02.983137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:03.052883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:03.075874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 300/607] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:58:03.111309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:03.120425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690bfc7-738e-433d-a3a1-56b2b4232c6a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:03.177430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:03.178009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:03.178330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:03.178626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:03.179361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:03.179675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:03.181145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:03.202525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 308/608] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:03.207967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:03.330074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee43dc-92dc-47aa-9915-4b70d77ded33 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:03.378660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:03.379368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:03.379654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:03.379936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/5889d3606c904dac9b4674efed1f94a6/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:03.404621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=5889d3606c904dac9b4674efed1f94a6, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:03.406089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:03.467540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:03.498434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 301/609] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:03 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/5889d3606c904dac9b4674efed1f94a6/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:03.621917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc85b5c-702a-49f1-8d19-3ece321d84c5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:03.674417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:03.675074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:03.675377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:03.675655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:03.688885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:03.693712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:03.694600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:03.695886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 309/610] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:03.701120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:03.743440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:03.806017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf200f8d-55b6-411e-a756-0886572690cd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:03.841414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:03.842030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:03.842313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:03.842587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:03.843335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:03.844550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:03.846368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:03.855833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 302/611] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:03.861553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:03.882484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:03.999933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44290b03-e3d5-4a6a-86d4-a0e559ad4a5e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:04.048790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:04.049440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.049726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.050005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.061243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:04.063438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:04.079429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:04.080392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:04.081666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:04.081666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:04.081666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:04.081666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 310/612] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:04.086676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:04.189097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:04.193745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025a63d-9003-4def-9a2a-70f1defcb023 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:04.231258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:04.231922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.232426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.232702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.233359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:04.233667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:04.235634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:04.248497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 303/613] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:04.254357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:04.334369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:04.376011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9c83-271e-4c37-932a-e0ee079dd5ab None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:04.427881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:04.428526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.428802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.429068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.437826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:04.439305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6c21dd59-ea7c-441b-923e-42d91c081b65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:04.450688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:04.451712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:04.453022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 311/614] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:04.458085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:04.566691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399e1ef-25e2-4918-99e4-d1013eb8fa31 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.566999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399e1ef-25e2-4918-99e4-d1013eb8fa31 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.567182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399e1ef-25e2-4918-99e4-d1013eb8fa31 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.568658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 312/615] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:04.581681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.581971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.582248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.611976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5b5b5-77b2-4718-9c72-abaaa99a20eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:04.614578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:04.615865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:04.679820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:04.680442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.680723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.680997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.681730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:04.682039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:04.683477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:04.710917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 304/616] 10.222.0.62 () {56 vars in 1135 bytes} [Tue Sep 13 09:58:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:04.720111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:04.726334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 313/617] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:58:04.746407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:04.856298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658ee2-8385-4e7d-9b9d-46eec03441a2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:04.890302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:04.891507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:04.891792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:04.892065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:04.902253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:04.903729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:04.918493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:04.919480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:04.921276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:04.921276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:04.921276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:04.921276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 305/618] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:04.925259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.018234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:05.069145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ecf46-4e24-4cc8-81cf-a636ba2a1eba None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.113486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.114062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.114348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.114620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.115289 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:05.115595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:05.116960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:05.129383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 314/619] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:05.136259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.148062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.251589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:05.314088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f180-a34f-458f-a256-4036dd5d9703 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.332337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077f27c-1c89-4edb-9722-f7adff853c58 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.353629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:05.361313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.362348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.366634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.366634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.373588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:05.373847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b8581-0313-4631-9dc8-b7fe3e3ea97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.374587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b8581-0313-4631-9dc8-b7fe3e3ea97c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.374972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b8581-0313-4631-9dc8-b7fe3e3ea97c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.375949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b8581-0313-4631-9dc8-b7fe3e3ea97c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.376574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:05.388370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b8581-0313-4631-9dc8-b7fe3e3ea97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:05.391747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:05.392839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:05.393971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-8e9b8581-0313-4631-9dc8-b7fe3e3ea97c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:05.399195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:05.400138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 315/620] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:05.400986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09214ca2-c76d-4d55-b2a9-6d8e4801df76 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.402271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09214ca2-c76d-4d55-b2a9-6d8e4801df76 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.402628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 306/621] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:05.404290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09214ca2-c76d-4d55-b2a9-6d8e4801df76 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.405109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.406075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 316/622] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:05.406934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09214ca2-c76d-4d55-b2a9-6d8e4801df76 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.519583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e187c6-a2d7-405d-affb-e87e4369b48e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.548287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09214ca2-c76d-4d55-b2a9-6d8e4801df76 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.559440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:05.566436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.567358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.575316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.575316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.575316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:05.575316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:05.575316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b274a6-5a2d-4220-9292-88116851500b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:05.586035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 307/623] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:05.587546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.588035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.588306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.594275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.595093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.595841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.596276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.598120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:05.598593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:05.600220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:05.608467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:05.609826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:05.621225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 317/624] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:05.621225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.669923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:05.692015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 308/625] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:05.693203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.802103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6860ffe-6139-41b6-bcfa-23ff27326669 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.817727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1734e6-287e-434f-8e67-c2b38783fe07 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:05.849090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:05.852549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.853178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.853454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.853722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.864589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:05.865422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402bdb-bdbe-4c88-b8c1-3d76932635dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:05.866820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:05.866933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402bdb-bdbe-4c88-b8c1-3d76932635dd None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.867191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402bdb-bdbe-4c88-b8c1-3d76932635dd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.867931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402bdb-bdbe-4c88-b8c1-3d76932635dd None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/174cca9647954ad1afdd4e6b8fba18f4/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-df418279-a9f0-4a36-8707-e4017d1f5151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:05.881638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:05.882577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:05.883495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:05.883495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:05.883495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:05.883495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 318/626] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:05.883723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199bf8a-75fc-4bae-8ba4-a0f978df22e0 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.884456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199bf8a-75fc-4bae-8ba4-a0f978df22e0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.884456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199bf8a-75fc-4bae-8ba4-a0f978df22e0 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.889886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59402bdb-bdbe-4c88-b8c1-3d76932635dd None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=174cca9647954ad1afdd4e6b8fba18f4, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:05.890025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 319/627] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:05.890793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199bf8a-75fc-4bae-8ba4-a0f978df22e0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:05.892141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59402bdb-bdbe-4c88-b8c1-3d76932635dd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:05.969849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 309/628] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:05 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/174cca9647954ad1afdd4e6b8fba18f4/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:05.969849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:05.969849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:05.969849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:05.991537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:05.992786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:06.025250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.043273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199bf8a-75fc-4bae-8ba4-a0f978df22e0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.083886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 310/629] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:06.085312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.100970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.101601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.101881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.102155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.102818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:06.103119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:06.104557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1cf8a-7fad-4ca3-9672-17f2c34f231a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:06.118608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 320/630] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:06.120153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a7c4-bea4-462a-a117-0d9c25b3aeb7 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.120439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a7c4-bea4-462a-a117-0d9c25b3aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.120708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a7c4-bea4-462a-a117-0d9c25b3aeb7 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.122193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 321/631] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:06.123592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2a7c4-bea4-462a-a117-0d9c25b3aeb7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.218344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bfdff-e0d1-440a-b32e-38f2014b2bab None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.221936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.251437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2a7c4-bea4-462a-a117-0d9c25b3aeb7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.256164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3004db7-2ead-4050-9a47-949560bb0277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.256801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3004db7-2ead-4050-9a47-949560bb0277 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.257088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3004db7-2ead-4050-9a47-949560bb0277 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.257365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3004db7-2ead-4050-9a47-949560bb0277 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.269460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3004db7-2ead-4050-9a47-949560bb0277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e3004db7-2ead-4050-9a47-949560bb0277 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:06.274312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:06.275388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:06.276215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 311/632] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:06.276931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.277377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.277482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.298931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.299968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.300464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.300957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.306793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:06.308325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:06.309102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:06.310627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:06.322126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:06.323205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:06.324168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 322/633] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:06.324762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.341624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.390488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 312/634] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:06.391739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.454278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d1d36-94b6-40ce-bfbb-f2231e395fa5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.491600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.492324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.492606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.492881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.493541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] RBAC: Adding query filter params (name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:06.493844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:06.495220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:06.504100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 323/635] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:06.505222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.516720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.525543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453debd-a464-4794-a873-89e51aca4d9f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.572647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.573743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.574449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.574885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.576149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:06.576609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:06.578866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:06.595573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.602812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 313/636] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:06.603773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.638618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f5f22-2ef3-43c3-b30f-46e7f89e309a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.682715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.683881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.684382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.684852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.696842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:06.702617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:06.703788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:06.704807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:06.706719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 324/637] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:06.707664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.713990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6d6b2-944e-4b90-b0a4-7eaa73143555 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.756411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.756957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.757235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.757503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.767981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:06.769426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:06.783912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:06.784878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:06.790944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:06.790944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:06.790944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:06.790944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 314/638] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:06.790944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:06.790944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.818321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42571409-95d0-4ceb-ac25-c7cf6c11c383 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.848062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:06.854249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.858954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.858954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.858954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.894802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68df0ab-5003-448f-95ab-9c6dbf8baa30 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:06.927734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:06.929207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:06.934192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:06.934901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:06.935206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:06.935486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:06.947038 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:06.952348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:06.953290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:06.954133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 315/639] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:06.954810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.016225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.046190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 325/640] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:07.047657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.063608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeb9dd-a857-40d4-a587-b0a44a5753cd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.077533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.106481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.107099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.107496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.107782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.108452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.108763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.110753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.124681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 316/641] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.124681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.158838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f03a3-7279-49ff-b72a-b7b2346053f0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.193497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.194476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.194853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.195104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.195713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.196020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.197395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.209129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 326/642] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.209924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.233559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c607f05-57fb-41b7-bd91-c6d43e17c23c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.258740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.270151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.270951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.271372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.271651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.272319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.272625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.274124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.288263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 317/643] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.288954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.308529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.320387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487616c-f302-4d58-93ed-f9649a01286c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.356025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.356658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.356932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.357372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.369889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.371356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:07.385675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:07.386694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:07.387658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:07.387658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:07.387658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:07.387658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 327/644] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:07.387658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.402951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934327e4-a13a-4557-b630-9dc0dcaa72c1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.440338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.441113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.441273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.441518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.451644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.453089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:07.472791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:07.473761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:07.480066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:07.480066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:07.480066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:07.480066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 318/645] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:07.480066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.506003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c9605-0883-4a6b-9d3e-4cd29e1ad3c6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.521348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.541645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f234-628d-481c-b853-0e407768930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.542130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f234-628d-481c-b853-0e407768930f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.542435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f234-628d-481c-b853-0e407768930f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.542702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f234-628d-481c-b853-0e407768930f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.550773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89f234-628d-481c-b853-0e407768930f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.552246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89f234-628d-481c-b853-0e407768930f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0c89f234-628d-481c-b853-0e407768930f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:07.563838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:07.564789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 328/646] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:07.565722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f234-628d-481c-b853-0e407768930f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.568943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.588698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e52d3-de61-4de6-977d-e67c687f3afe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.629040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.629669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.630043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.630347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.643139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.644610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:07.663002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:07.664251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:07.665131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:07.665131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:07.665131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:07.665131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 319/647] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:07.665131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.676584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f234-628d-481c-b853-0e407768930f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.718501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.719147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.719455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.719734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.720378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.720685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.722060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.734721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 329/648] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.735399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.765452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.781547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1425a5-486d-4358-8fd3-a4e595176333 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.805705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:07.811677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.812385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.813165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.813435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.814080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.814384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.815801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.832398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 320/649] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.832398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.850496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840b79-b1fc-45e1-aadb-d0e385f6d9ef None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.887424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.888040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.888323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.888597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.889327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.889636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.891724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.909302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 330/650] 10.222.0.62 () {56 vars in 1137 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.910208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:07.941383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cf007-410d-4c6d-91ec-a4bb6b0a90c7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:07.976611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:07.977216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:07.977499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:07.977768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:07.978403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:07.978712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:07.980089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:07.992126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 321/651] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:07.992759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.005659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.018875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3011e3-23f8-4f18-8013-d415a03e1d5c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.040161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.054422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.055021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.055314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.055594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.062209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.064211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:08.075611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:08.076697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:08.077831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:08.083317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 331/652] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:08.084500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.105302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d5219-6a2f-4380-b532-77dbf1c997c7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.146811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.147479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.147765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.148035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.155327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.156797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:08.168471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:08.169419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 322/653] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:08.170369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.209830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6949e83-2faa-4313-b8d3-9578e5a10adc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.243445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.244060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.244343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.244617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.245986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.276181 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32588fda-398c-439a-a80d-2a7169203859 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.293460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.310609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.311266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.311552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.311826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.313262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.314549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.319695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.321110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:08.332171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:08.333100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 323/654] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:08.334025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.411366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 332/655] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:08.411366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.441845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168947f-caf3-4439-b16a-a954a4cf2429 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.469988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.475845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.476442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.476719 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.476988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.477664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:08.477965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.479392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b7692-1be8-416f-8c51-04740a672517 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.492236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 324/656] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:08.493368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.493561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0320-fa0c-4a02-8ddc-a326615fe92e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.493846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0320-fa0c-4a02-8ddc-a326615fe92e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.494160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0320-fa0c-4a02-8ddc-a326615fe92e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.495634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 325/657] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:08.500551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0320-fa0c-4a02-8ddc-a326615fe92e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.512322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef76e0c-b122-47b1-b1d6-e1cb5ed48087 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.548601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.549128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.549411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.549679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.550579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:08.550698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.552164 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.569193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 333/658] 10.222.0.62 () {56 vars in 1137 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:08.569987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.605609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0320-fa0c-4a02-8ddc-a326615fe92e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.639555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.640156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.640437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.640712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.641512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:08.641821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.643269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.661184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 326/659] 10.222.0.62 () {56 vars in 1139 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:08.662143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.669780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e4d89-8349-4ffa-95a6-da9cf1b1cde5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.702945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.703617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.703896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.704162 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.710428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.710553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.714434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.715940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-00ac3f4f-bfec-4094-820f-a2a7cf9e76f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:08.730720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:08.731943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:08.733098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:08.733098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:08.733098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:08.735505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 334/660] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:08.737088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.737373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.737705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.755091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:08.756312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:08.764258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e2dd4-e2c5-4847-95f9-8cf6c404a526 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.797869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.798461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.798802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.799022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:08.799772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=81eb5931848749a1ad22d9d228899cda) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:08.800087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:08.801499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:08.807286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 327/661] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/role_assignments?user.id=81eb5931848749a1ad22d9d228899cda&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:08.808057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.828761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 335/662] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:08.829658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:08.957051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f436c-e32a-4d05-bd3f-534b8b2b1c94 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.971741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:08.987665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a2b49-4c60-4a1a-9768-313c605c684a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:08.996975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:08.997580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:08.997857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:08.998123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.009918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.029267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.030090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.030349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.030635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.080551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.082972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.122523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.124699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.195856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 328/663] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:09.196708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.214148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.240083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 336/664] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:09.240083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.260727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.310609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea8840-33de-4a12-8829-3a4292ccf8d4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.346257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.346607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a87a5-ca74-41ba-b296-627ceea48cd7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.346785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.347068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.347369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.348071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:09.348311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.349930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.366045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 329/665] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:09.366045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.393036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.393776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.394146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.394478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.405339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:09.410738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:09.411889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:09.412941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:09.413621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 337/666] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:09.414074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.449334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.485197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b80a3a-72b0-4aa9-9ffd-4e9195af427d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.500417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.519911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea16ff-ab1c-4501-8c8e-26cdc6fdc9c4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.522529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.523207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.523494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.523766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.535433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.536867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:09.551678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:09.552613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:09.553504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:09.553504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:09.553504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:09.557408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 330/667] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:09.558274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.559133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.559798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.560080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.560354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.571510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.572957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:09.588135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:09.589286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:09.590360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:09.590360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:09.590360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:09.594635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 338/668] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:09.595505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.670312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599634d-62fd-4813-b517-4b72670bf904 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.696182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.704815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9aaff-7ddc-4479-aa20-c247f4785961 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.707893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.708605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.708974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.709323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.716456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.717856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:09.730831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:09.731921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:09.732963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:09.739985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 331/669] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:09.739985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.743529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.748214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.749241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.749241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.749559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.750261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] RBAC: Adding query filter params (name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:09.752905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.752905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.760616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 339/670] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:09.761370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.853871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adc2c9-3b97-4365-b179-c5ef45216f37 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.869946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721327b-cc3f-42f1-a6c0-3e291d361624 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:09.889283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.890100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.890521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.891472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.892266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:09.892707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.894192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.907404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 332/671] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:09.907855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:09.908259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.908872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:09.909158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:09.909431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:09.910074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:09.910380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:09.911762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:09.923773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 340/672] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:09.924359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:09.964279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:09.985883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.023014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ac2fa-02a7-40eb-8db9-e117c3013425 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.031097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5e116-7e89-4860-b72c-3b0cd5920fbe None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.061208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.062214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.063199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.063644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.064464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:10.064925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.065833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.066465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.066751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.067034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.067143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.078686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.080171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.083338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 333/673] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:10.084276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:10.099350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:10.105367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:10.106245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:10.106245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:10.106245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:10.106245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 341/674] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:10.106245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.195269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fba37c-e379-48d7-a20b-332e7c8d9b1f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.198505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.210444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876a19-d436-4d28-ad0b-7232c80471f1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.218049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.232569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.235412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.235412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.235412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.242533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.244002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.248531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.249164 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.249448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.249722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:10.255478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:10.256548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:10.257626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:10.259427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.259600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 334/675] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:10.260499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.261354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:10.273145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:10.274220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:10.275359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:10.281789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 342/676] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:10.281789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.372654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce9b9f-5a06-4f0c-b166-4e52d9c27aa9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.393142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8b63-d038-40d1-a087-d4cf3f96fb99 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.410322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.410995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.411374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.411705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.412491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=5950076b981b43feba3185f1fb513729) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:10.412876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.414346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.420499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 335/677] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/role_assignments?user.id=5950076b981b43feba3185f1fb513729&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:10.421311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.442591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.443569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.444586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.445092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.446290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:10.446761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.449285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.451292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.467427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 343/678] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:10.468322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.532757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.545983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a034b-fedf-41b6-8a28-4bbf2436855a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.598078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeb58f-ad48-425f-b5d9-7bc4deab50f5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.600225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.602078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.602535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.602963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.603964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:10.604427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.606059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.623156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 336/679] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:10.624400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.644134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.644924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.645301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.645747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.646539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:10.646850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.648343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.662768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 344/680] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:10.663966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.740360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.765417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7252b-306a-4d6e-8654-362a32de7286 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.781537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:10.799092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c52ef-b859-4de1-b79c-48d35c47d059 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:10.806680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.807430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.807782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.808507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.817842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.819410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:10.834285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:10.835326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:10.836281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 337/681] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:10.836888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.838205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:10.838808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:10.839081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:10.839378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:10.840187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=566f088827ab4710bf54d64415831a1c) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:10.840399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:10.841818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:10.859646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 345/682] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/role_assignments?user.id=566f088827ab4710bf54d64415831a1c&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:10.860272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:10.962476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c69b7-490b-4d5d-9821-15c6564baa63 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:11.003323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:11.004753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.005238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.005788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.006760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=5889d3606c904dac9b4674efed1f94a6) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:11.007313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:11.008982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:11.017088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 338/683] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/role_assignments?user.id=5889d3606c904dac9b4674efed1f94a6&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:11.051017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94e2f0-414f-47eb-8302-5e454afdce6a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:11.106537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:11.107604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.108257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.108735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.110324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:11.110893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:11.112744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d4a0-30a6-4cd3-bc1d-a48d18524b61 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:11.134785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 346/684] 10.222.0.62 () {56 vars in 1135 bytes} [Tue Sep 13 09:58:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:11.140401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:11.152446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b55ed-f8b2-42b1-9905-aeab25953eb6 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.156018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b55ed-f8b2-42b1-9905-aeab25953eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.156678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b55ed-f8b2-42b1-9905-aeab25953eb6 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.158385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 347/685] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:11.164697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.165559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.166065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.169946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:11.192829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:11.194341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:11.203536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:11.284188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b936bf9-1ede-4171-a975-26443342ce20 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:11.286736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 348/686] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:11.321284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:11.321910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.322330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.322605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.393271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:11.394733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:11.408845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:11.501393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 339/687] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:11.503768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:11.628157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5d702-a373-4177-bb52-fffc312a2199 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:11.650254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:11.667704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:11.668815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.669113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.669393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.678044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:11.682408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:11.683896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:11.702185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:11.703764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:11.704686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:11.704686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:11.704686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:11.707744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 349/688] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:11.710499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:11.826832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85492b-df59-4870-94f1-97ce1c439c79 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:11.863478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af98b9-2770-4042-b4ec-42c0c5907af0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:11.884353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:11.894523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:11.895144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.899863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.899863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.899863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:11.899863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:11.899863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:11.918315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:11.918972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:11.919282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:11.919560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:11.929437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 350/689] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:11.935743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:11.949296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:11.956816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:11.957727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:11.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:11.963209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 340/690] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:11.968395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.038608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:12.101331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06230f7-bb60-4d7c-a192-a369c54a8de5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.143323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113c6c-b3b1-4eb3-a6d2-cd76273d3b97 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.145485 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.146098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.146380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.146654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.148318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:12.148633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.150007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.168146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 341/691] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:12.174448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.211583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:12.211971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.212634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.212922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.213200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.221011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.222523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:12.242899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:12.244969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:12.247077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:12.252616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 351/692] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:12.258168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.282571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467781b4-43c6-4ffa-a8a2-aeb1ad4c54c1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.320323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.321132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.321420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.321802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.333441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.334966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:12.351196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:12.353531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:12.355675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:12.355675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:12.355675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:12.355675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 342/693] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:12.358088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.451924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:12.469316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bc66-7e43-4f04-b94c-c2b621ac6202 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.501403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70da07-dbfb-4e8b-bb0f-77a51cbae9d0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.509041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.509759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.510049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.510326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.510977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:12.511316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.513926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.517557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:12.527922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 343/694] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:12.533990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.568580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.569678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.569962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.570238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.570936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:12.571271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.572757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.587009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 352/695] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:12.596870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.733188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00955fcd-5d82-40b5-b3e4-bfc596135dd1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.757693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866b673-1d7f-4131-8a0d-e9a32606d1c3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:12.787853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.788467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.788748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.789021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.789750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=174cca9647954ad1afdd4e6b8fba18f4) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:12.790062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.791794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.804238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 353/696] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:12 2022] GET /identity/v3/role_assignments?user.id=174cca9647954ad1afdd4e6b8fba18f4&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:12.829944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:12.830616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:12.830900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:12.831192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:12.844645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:12.846142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:12.863075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:12.871559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:12.872512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 344/697] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:58:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:12.873087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:12.932079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:13.033576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4d636-7378-4a56-86f7-e6f2ea4666bc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:13.073829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:13.074445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.074771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.075049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.076020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:13.076329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:13.077767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b697-51a2-4845-85b2-a3c1a3ebde0f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:13.099647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 354/698] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:58:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:13.105316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:13.119226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:13.179401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a23832-835c-4e1c-861a-df15ec2c01cc None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.179694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a23832-835c-4e1c-861a-df15ec2c01cc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.179970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a23832-835c-4e1c-861a-df15ec2c01cc None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.181595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 355/699] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:13.187247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.187579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.187837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.211856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:13.213060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:13.227768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664b69-dbd5-481e-b024-b30ecdaa7ed2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:13.265947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:13.266592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.266877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.267148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.300608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 356/700] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:58:13.348716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:13.350233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:13.358423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:13.457735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 345/701] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:13.463053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:13.495464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:13.579145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3533cbd-d7a4-4c7b-a1ec-6e1ef13dfce8 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:13.615114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:13.615782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.616109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.616387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.626802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:13.628277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6f388e99-1bb0-4e57-8e18-5ecfc0dcd023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:13.643715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:13.644731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:13.645595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:13.645595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:13.645595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:13.645595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 357/702] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:13.650548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:13.668085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a66c-7fde-4f1b-8bac-9c3302f0ba43 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.668392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a66c-7fde-4f1b-8bac-9c3302f0ba43 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.668659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a66c-7fde-4f1b-8bac-9c3302f0ba43 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.670724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 358/703] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:13.676699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.677269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.677541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.697684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:13.698884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:13.771219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 359/704] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:13.779714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:13.780734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ba0f4-2949-409d-8da1-5fb591b55a73 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:13.783869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:13.789854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:13.818463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:13.819078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:13.819415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:13.819692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:13.820405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:13.820719 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:13.822075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:13.834193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 346/705] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:13.841169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:13.933209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5659fb-22ca-4932-9c96-918c4274b8fb None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.000515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.001728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.002021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.002298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.014568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.016599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-13135cd6-358b-4b5e-9803-55594de1d87a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:14.034663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:14.035706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:14.036615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:14.036615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:14.036615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.036615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 360/706] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:14.037330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906fe0-779f-40c3-9303-f9a5879e4b0d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.037620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906fe0-779f-40c3-9303-f9a5879e4b0d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.037893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906fe0-779f-40c3-9303-f9a5879e4b0d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.039684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 361/707] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:14.043882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906fe0-779f-40c3-9303-f9a5879e4b0d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.054192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788dcb1-9adb-426d-bc36-abc5b80f8c70 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.092722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.093559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.094479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.095052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.103241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.104951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.110414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c961f162-de7f-460e-9aa9-677bfc3474a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:14.118389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:14.119843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:14.121003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:14.121899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 347/708] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:14.123279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.126043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.126043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.143579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:14.144990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:14.166635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906fe0-779f-40c3-9303-f9a5879e4b0d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.172386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:14.202322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.203463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.203749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.204215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.204656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:14.204959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.206325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.224664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 362/709] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:14.224664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.227866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 348/710] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:14.231238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.329980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cc9f-5c17-4220-b2a4-f725eb6732a9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.345419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8efed-d7e8-4f0f-8237-8a40c488fa0e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.352170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:14.368264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.372315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.372315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.372315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.372315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.372315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:14.384572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:14.385729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:14.386572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.389642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 363/711] 10.222.0.62 () {56 vars in 1129 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:14.391662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.520130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.520928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.521278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.521617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.522380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:14.522748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.523531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edba41f-3b5e-45f8-ae6d-c331f9b534ed None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.524376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.543245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 349/712] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:14.544257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.563063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:14.565182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.565790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.566069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.566339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.566980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.568440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.644644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f4ef1-6fa4-415c-a581-54ec7129c642 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.672702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 364/713] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:58:14 2022] POST /identity/v3/users => generated 254 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:14.678794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.679455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.679734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.680002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.680654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:14.680953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.682371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.684373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.696090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 350/714] 10.222.0.62 () {56 vars in 1124 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:14.696090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.745759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:14.793248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:14.816165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc1a10-31f4-4ca7-a40c-8c3d2b1bab35 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.847593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175848e2-c944-4f8d-8b42-2bb8fb543ba2 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:14.857303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.858092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.858468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.858842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.860299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=81eb5931848749a1ad22d9d228899cda) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:14.860650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.862203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.868896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 351/715] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/role_assignments?user.id=81eb5931848749a1ad22d9d228899cda&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:14.869638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:14.898146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:14.898730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:14.899010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:14.899489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] PATH_INFO: `/v3/services/network` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:14.899997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:14.901523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:14.912657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:14.913914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:14.914760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 365/716] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:14.921280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:15.051732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666820f7-91c7-4ee5-9997-ef31a5d17e3d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:15.056366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:15.059736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef98b6d-40c0-495a-b8d0-b4385398f35e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:15.093795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:15.095347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:15.095808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:15.096350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:15.097271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:15.097576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:15.098356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:15.098943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dde630-37d1-44a7-8113-b920ac5f0a0f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:15.099048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:15.099335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:15.099615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:15.100342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] RBAC: Adding query filter params (name=network) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:15.100666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:15.102106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:15.107927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 366/717] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:15.111316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 352/718] 10.222.0.62 () {56 vars in 1124 bytes} [Tue Sep 13 09:58:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:15.119642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:15.234808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec305-62ea-47eb-aeab-b3231d0aec0e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:15.279175 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:15.279810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:15.280096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:15.280369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:15.281550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:15.281914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:15.283374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c21a1f-cac3-45a0-accd-b1fad52ffca9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:15.290052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 367/719] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:15.559705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046b153-8f24-4207-ac66-dfaddcc12d72 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:15.560694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046b153-8f24-4207-ac66-dfaddcc12d72 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:15.561191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046b153-8f24-4207-ac66-dfaddcc12d72 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:15.563027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 353/720] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:15.568682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:15.569210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:15.569689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:15.589662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:15.591313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:15.617138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:15.677786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 368/721] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 09:58:16.075134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046b153-8f24-4207-ac66-dfaddcc12d72 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:16.087578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:16.185179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046b153-8f24-4207-ac66-dfaddcc12d72 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:16.221146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:16.221781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:16.222060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:16.222331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:16.233087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:16.238019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:16.238960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:16.239832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 354/722] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:16.245150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:16.391080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0727c2-3dae-4dfd-96bc-18825ac66925 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:16.446929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:16.447594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:16.447880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:16.448157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:16.448838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:16.449154 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:16.450539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:16.467642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 369/723] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:16.476259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:16.592636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:16.635621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267fb2c-4f69-4744-b05d-e4a927f81176 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:16.680070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:16.680629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:16.680918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:16.681195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:16.696780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:16.698216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:16.715670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:16.716708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:16.717576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:16.717576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:16.717576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:16.717576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 355/724] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:16.723013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:16.755052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:16.852237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634c54-7121-4802-b48b-774a568cd4b7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:16.892853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:16.893846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:16.894407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:16.894877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:16.895741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:16.896260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:16.897825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:16.911363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 370/725] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:16.919196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:17.071284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b6ae2-f3b0-47f4-bc46-7ff32a349388 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:17.100730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:17.122456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:17.123573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.124020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.124456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.135047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:17.136809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f51652e5-74c6-4d6e-aeaa-036dea4cb77b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:17.149631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:17.150534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:17.151552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 356/726] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:17.157250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:17.288001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:17.308622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b03b5-1aee-412e-a42b-b336c4e7b66a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:17.363430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:17.363963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.364250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.364527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.365258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:17.365567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:17.366994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:17.389141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbee0-8395-4769-9b7e-8749e4d6178a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.389467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbee0-8395-4769-9b7e-8749e4d6178a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.389698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbee0-8395-4769-9b7e-8749e4d6178a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.391605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 357/727] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:17.396997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f837cd-a685-478b-abef-9b4cb1b73c0e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.397510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f837cd-a685-478b-abef-9b4cb1b73c0e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.398005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f837cd-a685-478b-abef-9b4cb1b73c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.404797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 371/728] 10.222.0.62 () {56 vars in 1135 bytes} [Tue Sep 13 09:58:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:17.408864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:17.433941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-37f837cd-a685-478b-abef-9b4cb1b73c0e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:17.435140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-37f837cd-a685-478b-abef-9b4cb1b73c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:17.563352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 358/729] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:17.609924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8085be-d19a-4557-8ee8-3f26086f6088 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:17.670184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:17.670745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.671030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.671335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.685972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:17.778605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:17.780106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:17.789697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ecfc-0e2e-4109-bca2-6ee93079110b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.790487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ecfc-0e2e-4109-bca2-6ee93079110b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.790569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ecfc-0e2e-4109-bca2-6ee93079110b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.793502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 359/730] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:17.798760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:17.798760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:17.798760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:17.817304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:17.820905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:17.909898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 360/731] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:17.915550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:17.928285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 372/732] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 09:58:17.933793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:17.990609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:17.991861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.059843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b76ab-851a-42c7-96b8-416d7f6e8ef4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.073097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40a9d9-3a2f-4744-b93c-db34b31807a1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.100008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f51b0-f3f8-4895-b853-5e13f5d1985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.100942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f51b0-f3f8-4895-b853-5e13f5d1985a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.101558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f51b0-f3f8-4895-b853-5e13f5d1985a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.101862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f51b0-f3f8-4895-b853-5e13f5d1985a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.114021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.114376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f51b0-f3f8-4895-b853-5e13f5d1985a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:18.114870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.115286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.115663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c08f51b0-f3f8-4895-b853-5e13f5d1985a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:18.119364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:18.120329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:18.121471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 361/733] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:18.122312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c6e7-6906-42f8-ac0e-c5881c82ab76 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.122312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c6e7-6906-42f8-ac0e-c5881c82ab76 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.122602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c6e7-6906-42f8-ac0e-c5881c82ab76 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.128255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 362/734] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:18.129103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c6e7-6906-42f8-ac0e-c5881c82ab76 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.133900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.135664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d57ec309-d7c6-4aca-8c9b-630ae7fe6e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:18.153560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:18.154575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:18.155585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:18.155585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:18.155585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:18.155585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 373/735] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:18.156309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.156605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.156883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.175764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:18.176968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:18.245063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.246336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 374/736] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:18.247084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.249025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c6e7-6906-42f8-ac0e-c5881c82ab76 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.267842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.286347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.286959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.287270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.287550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.288213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:18.288522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.289919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04c100-67c8-41a8-ad78-0381aa91bc9c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.299243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 363/737] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:18.300484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f7893-7e09-4ad9-acd7-58349ab8cfc8 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.300777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f7893-7e09-4ad9-acd7-58349ab8cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.301535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f7893-7e09-4ad9-acd7-58349ab8cfc8 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.307267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 364/738] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:18.308056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f7893-7e09-4ad9-acd7-58349ab8cfc8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.359855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ecf54-67f2-4f11-8002-4f7bccee4d52 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.402719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.403655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.403941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.404215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.404862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:18.405169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.407496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.419808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f7893-7e09-4ad9-acd7-58349ab8cfc8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.423335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 375/739] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:18.424151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.460117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44597fbd-f40b-45ab-9480-9db393daac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.460742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44597fbd-f40b-45ab-9480-9db393daac12 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.461023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44597fbd-f40b-45ab-9480-9db393daac12 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.461296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44597fbd-f40b-45ab-9480-9db393daac12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.471954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44597fbd-f40b-45ab-9480-9db393daac12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-44597fbd-f40b-45ab-9480-9db393daac12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:18.476633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:18.477532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:18.478379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 365/740] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:18.479046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.479120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.479708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.498433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:18.499799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:18.518203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.520332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.538812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd54036-dc23-43cf-bcaa-d107484e2e7d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.568223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 366/741] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 09:58:18.569051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.579326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.580015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.580305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.580581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.592450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.594146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:18.610243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:18.611213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:18.612698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:18.612698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:18.612698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:18.612698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 376/742] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:18.612698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.681963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5146abf-ac75-4a3a-af7e-d64e05bacfb6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.715105 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4293b9c-cb21-477e-9f4f-6face70499f5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.720344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.720971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.721252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.721520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.728398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.729836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:18.741268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:18.742149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:18.743047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 367/743] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:18.744017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.752038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.756988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.757596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.757924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.758205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.758868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:18.759253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.762309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.762408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.769420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 377/744] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:18.770211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.857536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882b573-6332-489a-acdc-53f6c5fa6d57 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.877005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46ef6-f887-4352-81bc-e39a4e80ff5d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:18.890670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.891301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.891580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.891852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.892486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.893823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.910375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:18.910974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:18.911292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:18.913852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:18.913852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:18.913852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:18.913852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:18.925534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 378/745] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:18.926307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:18.979320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:18.980959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 368/746] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:18 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:18.981782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.029029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cfac2-47bd-48e8-81fe-fffba1bee3a3 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.066206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:19.073741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.074589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.074874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.075151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.076088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:19.076297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.077768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.098961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 379/747] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:19.098961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.111435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7b4f5-da39-40e5-a107-96e88a4fc817 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.150325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.151338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.151763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.152203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.164814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.166528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:19.185183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:19.186174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:19.187117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:19.187117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:19.187117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:19.187117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 369/748] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:19.191960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.208793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21e272-9440-4e9d-9d65-e477640551f3 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.222153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:19.250200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.251040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.251527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.251960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.252618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.254068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.297542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c001b93-1e3e-457d-895e-eefd92ef4afc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.320127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:19.322726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 380/749] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:19 2022] POST /identity/v3/services => generated 240 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:19.331968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.360988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.361638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.361915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.362185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.371880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.374743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:19.388272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:19.389319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:19.390787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 370/750] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:19.391938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.475836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad82e4-e419-4f6d-89bf-0e5955954bf8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.518529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041ca81-0f96-4140-938b-a4e3c7b9e006 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.527315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.527938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.528229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.528507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.529247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=174cca9647954ad1afdd4e6b8fba18f4) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:19.530332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.531581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.537773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 381/751] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/role_assignments?user.id=174cca9647954ad1afdd4e6b8fba18f4&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:19.538594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.568336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.569307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.569749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.570260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.571078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:19.571595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.573951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.577074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:19.590283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 371/752] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:19.598833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.660497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:19.667247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ec833-347c-4acf-9d4b-91a1ff10bedc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.733746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.734584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.735226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.735817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.736991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:19.737390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.739207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.740289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a99d3-bef1-4e4e-b17e-efd580014419 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:19.776017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 382/753] 10.222.0.62 () {56 vars in 1149 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:19.778329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.796324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:19.797154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:19.797536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:19.797939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:19.809325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:19.810976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:19.830164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:19.831468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:19.832625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:19.838607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 372/754] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:19.844990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:19.923905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:19.944052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6f94e-dc71-4c62-a8b5-a2e2b5605a6b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.015771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.016688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.016975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.017245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.056337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:20.061406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03961119-128e-4bc0-a8a2-dc3d17406793 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.130008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.131621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.140793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.141403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.141680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.141942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.143782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:20.145862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.147451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.178882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 373/755] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:20.187771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:20.301808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:20.314882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 383/756] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:58:20.321217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:20.378979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b660f-77fd-4f72-851f-56ac0a6efb2d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.441614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.442767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.443045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.443341 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.472263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.477083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.505170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b8e4-5066-4ed9-8d3f-50600692f223 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:20.505804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:20.506887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:20.508006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:20.508006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:20.508006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:20.510281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 374/757] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:20.516418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:20.539501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:20.574739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.575430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.575715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.575987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.597157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.598628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:20.632224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:20.633245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:20.634165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:20.634165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:20.634165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:20.634165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 384/758] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:20.638930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:20.642628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade6851-e0b7-4e83-8596-41b14aaf381a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.677989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.678517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.678820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.679067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.679738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:20.680043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.681616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.694507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 375/759] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:20.700236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:20.751528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:20.834133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c892adc-05d1-4a30-8c8f-06561b0633ca None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.843571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:20.879216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.879841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.880128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.880403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.881052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:20.881357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.882502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2853d12-943f-415e-b948-d71af957f947 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:20.882733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.909484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 385/760] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:20.915022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:20.918300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:20.918982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:20.919300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:20.919623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:20.927402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:20.929290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:20.941828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:20.942913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:20.944034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:20.945883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 376/761] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:20.950984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:21.065381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639da28-be61-4518-81ab-ef634c34bd4f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:21.096707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:21.102720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:21.103445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.103781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.104110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.104922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:21.105287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:21.107032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:21.121282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 377/762] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:21.127053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:21.129399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e71b-2b31-4618-9f0d-4be7b1489c40 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:21.136175 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:21.215591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:21.216144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.216419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.216684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.234457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:21.235941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:21.256717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:21.257799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:21.258844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 386/763] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:21.261454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c46f0e-4471-41bb-914e-072456bd401d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:21.264671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:21.321565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11bd24-ed5f-476c-b89e-8feef45e5cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:21.322245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11bd24-ed5f-476c-b89e-8feef45e5cea None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.322523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11bd24-ed5f-476c-b89e-8feef45e5cea None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.322794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11bd24-ed5f-476c-b89e-8feef45e5cea None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/81eb5931848749a1ad22d9d228899cda/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.327191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11bd24-ed5f-476c-b89e-8feef45e5cea None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=81eb5931848749a1ad22d9d228899cda, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:21.328696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11bd24-ed5f-476c-b89e-8feef45e5cea None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:21.415090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 378/764] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:21 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/81eb5931848749a1ad22d9d228899cda/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:58:21.416906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c343dc-53f3-4266-931e-b7b1d3df9dcf None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:21.437079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962be7-2759-4569-8c5d-e135d4910190 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.437376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962be7-2759-4569-8c5d-e135d4910190 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.437795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962be7-2759-4569-8c5d-e135d4910190 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.439463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 379/765] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:21.445231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.445569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.445925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.462940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:21.463957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.464491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.464935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.465270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:21.466144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:21.466610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:21.466826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:21.469071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:21.487903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 387/766] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:21.500530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:21.523689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:21.549389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 380/767] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:21.563651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:21.662613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db942-d924-4275-9b2a-adb718db2ec2 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:21.698414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:21.699023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:21.699336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:21.699613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:21.700405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=4c062a52bd70482188def01c1ed57ab2) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:21.700772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:21.702222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5efdc-d527-45e5-b982-220e7d79c829 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:21.710133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 388/768] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:21 2022] GET /identity/v3/role_assignments?user.id=4c062a52bd70482188def01c1ed57ab2&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:21.878210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.009698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08abf10-e7da-41ec-bb00-411f0f68a50d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.021366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6df59-29a6-41c2-b9b2-4faba2b00eca None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.021599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6df59-29a6-41c2-b9b2-4faba2b00eca None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.021818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6df59-29a6-41c2-b9b2-4faba2b00eca None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.024889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 389/769] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:22.032752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0752c-b61d-44f2-9d08-2c4eb5c6abb1 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.033075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0752c-b61d-44f2-9d08-2c4eb5c6abb1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.033309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0752c-b61d-44f2-9d08-2c4eb5c6abb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.051048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.052325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.053029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.053502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.054424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:22.056042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:22.069828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:22.080080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:22.080885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 381/770] 10.222.0.62 () {56 vars in 1129 bytes} [Tue Sep 13 09:58:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:22.081658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-eab0752c-b61d-44f2-9d08-2c4eb5c6abb1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:22.084426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-eab0752c-b61d-44f2-9d08-2c4eb5c6abb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:22.092583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.150758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:22.189907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 390/771] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:22.195199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0bb5f-f452-4b15-ba8b-2c26177f8fbd None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.195199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0bb5f-f452-4b15-ba8b-2c26177f8fbd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.195199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0bb5f-f452-4b15-ba8b-2c26177f8fbd None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.195199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 391/772] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:22.198911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.199189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.199475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.237068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:22.237261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f021d-6fae-444a-9308-ec8cc841cabc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.238287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:22.273236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:22.275187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.275809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.276092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.276365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.277047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:22.277354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:22.278714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:22.284286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 382/773] 10.222.0.62 () {56 vars in 1124 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:22.289511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.412397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 392/774] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:22.443071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb6c71-f1fe-410b-84e9-dcffd48e883a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.460749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:22.467399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.501087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.501709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.501990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.502263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.502896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:22.503217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:22.504595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:22.513725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 383/775] 10.222.0.62 () {56 vars in 1124 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:22.522338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.654378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a110820-3b1c-467d-9efa-1eee76b4a1ce None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.708179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a3bf1-d0bf-43c8-bb5a-7a4d5da140f6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.720537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.721179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.721457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.721724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.722309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:22.725250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:22.739805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:22.740776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:22.741583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:22.747976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 393/776] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:22.752003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.774696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.775746 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.776026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.776296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.777126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] RBAC: Adding query filter params (interface=public, service_id=e4d616041d584d70895d8aaac47e4963, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:22.777440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:22.778863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:22.781393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:22.793399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 384/777] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/endpoints?service_id=e4d616041d584d70895d8aaac47e4963&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:22.797019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.831635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:22.891574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ffc6-6f51-4f8e-9801-c201c58224c6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.912715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7752-96dd-4904-9c45-e5e97fe305b7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:22.936740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.937391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.937672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.937942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.938570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] RBAC: Adding query filter params (name=network) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:22.939344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:22.941030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:22.946200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 394/778] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:22.947264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:22.960639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:22.961513 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:22.962062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:22.962550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:22.974824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:22.980439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:22.981362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:22.982195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:22.985418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 385/779] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:22.986368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.031812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:23.057474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eaac8-7415-4983-aa38-ba54ac73f125 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.097202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.098067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.098353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.098627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.099789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:23.100508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.102174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.108255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf44983-09c8-4f6d-9f3a-0296c1271677 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.111245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 395/780] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:23.114995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.240562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.241188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.241470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.241744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.242390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] RBAC: Adding query filter params (type=network) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:23.242698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.244117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.255439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 386/781] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:23.261454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.341990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef64fab-ce2a-42aa-811b-1b6fabab9efc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.378472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.379195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.380404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.380689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.381357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:23.381685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.383050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.394067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 396/782] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:23.395390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf761c6d-7133-4e9c-b1ab-7a9f38342cda None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.398718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.401261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:23.434340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.435383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.435868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.436358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.437774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] RBAC: Adding query filter params (interface=public, service_id=2d01188da9c54109a7b3d50f30d18562, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:23.438374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.439970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.446217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 387/783] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:23 2022] GET /identity/v3/endpoints?service_id=2d01188da9c54109a7b3d50f30d18562&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:23.451933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.479333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:23.527444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efed08-d2e8-4769-a02a-91ed82e13d62 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.563015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95206ad-bf4b-4107-a7da-cac3f9b8711c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.589634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.590742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.591373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.591848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.601031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.601031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.601031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.601031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.601403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.601994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.611199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 388/784] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:23 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:23.620399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.621884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:23.644171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:23.645180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:23.646081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:23.646081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:23.646081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:23.646081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 397/785] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:23.652768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.767184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:23.804011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dab559-b2fe-43eb-8e71-9f72aeff9d9d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:23.830738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:23.831413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:23.831693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:23.831963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:23.832619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:23.832919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:23.834375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:23.847670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 389/786] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:23.853614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:23.914953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:23.973150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9ea81-5536-465b-b64d-ead8385cd8c8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:24.020542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:24.021173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.021451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.021723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.033844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:24.035297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-6afbd38a-ae35-4c06-bdf1-496841b11e64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:24.046796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:24.047806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:24.048721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:24.057204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 398/787] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:24.062937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:24.213799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d619-d974-4bd1-bced-e66e99aed31f None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.213799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d619-d974-4bd1-bced-e66e99aed31f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.213799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d619-d974-4bd1-bced-e66e99aed31f None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.213799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 399/788] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:24.219041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75278f-1100-487e-956c-9745f9120559 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.219357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75278f-1100-487e-956c-9745f9120559 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.219636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75278f-1100-487e-956c-9745f9120559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.238181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-df75278f-1100-487e-956c-9745f9120559 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:24.239501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-df75278f-1100-487e-956c-9745f9120559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:24.282009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cd2eb-3982-46e6-b17c-29b499021ba7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:24.319558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:24.320507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.320991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.321552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.322506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 400/789] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:24.322763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:24.323302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:24.323808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104d9e5-3fdb-4306-b207-76345a01bc6c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.324097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104d9e5-3fdb-4306-b207-76345a01bc6c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.324368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104d9e5-3fdb-4306-b207-76345a01bc6c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.325550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:24.331284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 401/790] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:24.338102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.338384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.338659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.362891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 390/791] 10.222.0.62 () {56 vars in 1135 bytes} [Tue Sep 13 09:58:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:24.369163 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:24.375563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:24.377542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:24.379258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:24.447132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:24.501760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 402/792] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:24.548532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd55ed9-8ea1-46a4-956e-9fb091dcac57 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:24.594505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:24.595061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.595368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.595638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.614195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:24.615684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:24.615816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:24.635854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:24.636837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:24.638300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:24.638300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:24.638300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:24.638300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 391/793] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:24.643505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:24.735844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:24.781083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2eed6-f439-4047-87a9-7f2666a6ac29 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:24.788991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c39cc5-b70b-465d-ac45-42aba0555561 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:24.818188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:24.819281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.819785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.820257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.821108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:24.821740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:24.823516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:24.837654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 392/794] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:24.846413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:24.862853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:24.863526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:24.863805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:24.864078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:24.885578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:24.893117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:24.894045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:24.894889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 403/795] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:24.902725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:24.940693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.011848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d3321-051a-4460-8c97-f35d6bed18f0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.034140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd09e4b-ad36-4adf-aa8f-2fd78df46a61 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.048980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.049516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.049800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.050077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.050835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:25.051098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.052498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.061341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 404/796] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:25.062026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.067515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.079296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.080486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.080969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.081438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.090456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.092576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:25.108373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:25.110122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:25.111209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:25.113942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 393/797] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:25.115482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.183191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.189871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b0f5-3c3d-4f0e-add6-8be3a34e34fa None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.226088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.226876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.227287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.227663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.242902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:25.243406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3db94d-f580-4bd1-95b6-950a095ec0ac None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:25.249829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:25.250921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:25.251969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:25.259631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 405/798] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:25.259631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.287246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.287884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.288151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.288429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.301047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.302686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:25.320619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:25.321583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:25.322456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:25.322456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:25.322456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:25.326385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 394/799] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:25.327958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.359793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.406389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c54f9b-5f9c-46d8-8529-a3e9071b65a1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.433666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ca405-0abd-40ef-abc1-52448bb38920 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.454049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.455051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.455624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.456080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.456965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:25.457459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.459296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.470463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.471193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.471660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.471963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.472771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:25.473175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.474420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 406/800] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:25.474770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.475756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.488682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 395/801] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:25.488682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.500686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.586487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.590476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f29c4-18b9-4048-b3bd-ddafdbdbbe9e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.596982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b147ea1-b89d-4042-b7fb-239da7d6a977 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.626538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.627436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.627862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.628318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.629183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:25.629665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.631349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.634166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.634741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.635029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.635369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/174cca9647954ad1afdd4e6b8fba18f4/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.637809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=174cca9647954ad1afdd4e6b8fba18f4, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.639669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.647245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 407/802] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:25.648493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.713765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 396/803] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:25 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/174cca9647954ad1afdd4e6b8fba18f4/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:25.715033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.754306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:25.788381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747fbe8-4748-4384-91ed-a861dae944e1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.830161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.831039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.831215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.831403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.843144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.844756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.859661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13a1ea-3289-42c1-9ac3-e62478233fa4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-05b7813b-609f-438f-b9df-e7c48483ee56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:25.861701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:25.862627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:25.863578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:25.863578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:25.863578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:25.863578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 408/804] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:25.864587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8374-f8a9-4d62-ab28-49a68a59d031 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.864872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8374-f8a9-4d62-ab28-49a68a59d031 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.865344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8374-f8a9-4d62-ab28-49a68a59d031 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.871455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 409/805] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:25.872505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8374-f8a9-4d62-ab28-49a68a59d031 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:25.914311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:25.914864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.915417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.915689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.927000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:25.928470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-be5adb91-0aec-4b61-b762-2db8bac93e5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:25.943196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:25.944160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 397/806] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:25.945102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:25.945812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:25.975218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:25.976189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:25.987515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.008512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8374-f8a9-4d62-ab28-49a68a59d031 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.045821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.058046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.058829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.059281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.059666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.060481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:26.060891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.061615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 398/807] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:26.063048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.063194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.079295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 410/808] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:26.088184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.203773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca310a6-e5b5-4ba1-ae06-809957f24af8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.245740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.247046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.247375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.247651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.248637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:26.248946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.251718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32e56e-3edf-4aa1-aac8-31fb6e3b3a78 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.252855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a996f5-0519-4a47-95f4-64d79d7f83ea None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.269403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.278320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 399/809] 10.222.0.62 () {56 vars in 1149 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:26.278320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922123-37ed-4018-b8cc-8f794747569a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.278320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922123-37ed-4018-b8cc-8f794747569a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.278320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922123-37ed-4018-b8cc-8f794747569a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.278320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 400/810] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:26.281017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922123-37ed-4018-b8cc-8f794747569a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.311304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.313014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.313928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.314660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.324557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.326479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-e8c19cb6-c0e5-4145-b690-01820c47c3a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:26.344138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:26.345114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:26.346033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:26.348844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 411/811] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:26.350231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.350661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.351083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.411480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922123-37ed-4018-b8cc-8f794747569a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.451739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.452769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.453203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.453624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.465643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.467377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:26.484621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:26.485802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:26.486897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:26.486897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:26.486897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:26.492828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 401/812] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:26.493588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:26.494334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.494818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:26.540966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.553569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.564799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 412/813] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:26.565635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.625057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bb13-349a-40a8-b849-1947714e09b2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.668240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.669646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.670191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.670644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.672087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:26.672579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.674167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.679933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c2395-5cd9-41d7-bd06-fd5beac23b2f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.693193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 402/814] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:26.694284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.726183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.727044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.727541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.727821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.728407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.729763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:26.742840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:26.743872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 09:58:26.744716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:26.748838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 413/815] 10.222.0.62 () {56 vars in 1129 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:26.749709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.793535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.824076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9bd84-526f-416a-a315-24abe92b982b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.837879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:26.854046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1339c-9e89-4acb-a827-0d28dac94da4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:26.866618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.867268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.867555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.867833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.868479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:26.868785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.870409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.882405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 403/816] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:26.883007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:26.894447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:26.895057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:26.895489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:26.895772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:26.974319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:26.975798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:26.995381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ef5b-d9cf-4f4a-aedb-9f16fd1c2a82 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.021368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.030627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.034767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.034767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.034767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.034767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:27.034767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.034767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.044031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 404/817] 10.222.0.62 () {56 vars in 1124 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:27.044031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.079447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.085610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 414/818] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:58:27.091031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.148871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28356f05-9596-4487-abb9-85658e792c59 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.184779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.185409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.185694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.185971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.186548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.188489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.190178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb574e-c87a-4d6d-993e-bd75e77fee48 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:27.197200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:27.198377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:27.199336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 405/819] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:27.200124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.229873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.230500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.230780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.231048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.239058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.240520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:27.252291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:27.253144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:27.254030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 415/820] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:27.254538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.257335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.306312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.310350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a732abd-004f-4aa1-ad9c-e84cbbca92c3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.349230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.349912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.350126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.350398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.351013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:27.351374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.352876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.363195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 406/821] 10.222.0.62 () {56 vars in 1124 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:27.363348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78401bd7-f05b-455f-82e5-09caba7c2cc7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.363444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.397518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.398155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.398438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.398708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.410067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.411532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:27.426797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:27.427836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:27.428731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:27.428731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:27.428731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:27.428731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 416/822] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:27.428731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.478105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd902-f4d9-496f-8dc8-537be3141dc7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.487807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.512884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.513430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.513713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.514051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.514685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:27.515114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.516502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.525529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 407/823] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:27.525529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.539038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586617e6-a0d7-4b8e-bf51-c43a12b9a9aa None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.555173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.576028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.576635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.576921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.577195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.578525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:27.578835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.580454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.598956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 417/824] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:27.599857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.631493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eac28-4154-490f-8c3e-0bccd312def4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.666442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.667054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.667541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.667823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.668977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.670343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.702605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b5da0-5222-4d5c-a213-9d4a20d2470f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.716790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.736700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.737309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.737593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.737870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.738536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:27.738842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.740284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.752150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 418/825] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:27.753082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.762768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 408/826] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:27.763897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.864740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:27.907316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9648b7-0483-4749-be8e-8dee4c23cd3b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.907749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf0df-7a65-4080-878c-8b161b74dd8e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:27.946528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.947324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.947697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.947973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.948794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] RBAC: Adding query filter params (type=network) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:27.949087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfc0ef-8e73-4551-bb5c-82018e27ee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:27.949199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.949750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc0ef-8e73-4551-bb5c-82018e27ee03 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:27.950033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc0ef-8e73-4551-bb5c-82018e27ee03 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:27.950308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc0ef-8e73-4551-bb5c-82018e27ee03 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/4c062a52bd70482188def01c1ed57ab2/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:27.950528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:27.956370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 419/827] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:27.956370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:27.975318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfc0ef-8e73-4551-bb5c-82018e27ee03 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=4c062a52bd70482188def01c1ed57ab2, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:27.977028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfc0ef-8e73-4551-bb5c-82018e27ee03 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:28.000321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:28.064120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 409/828] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:27 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/4c062a52bd70482188def01c1ed57ab2/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:58:28.064303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c301b6-2b5d-4457-b7f7-7d58d788c082 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:28.064396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e424d-b2ef-4e81-866a-01f69a7a8f5a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.064396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e424d-b2ef-4e81-866a-01f69a7a8f5a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.064396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e424d-b2ef-4e81-866a-01f69a7a8f5a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.064396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 410/829] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:28.064396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e424d-b2ef-4e81-866a-01f69a7a8f5a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:28.110983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:28.111862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.112224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.112619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.123066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:28.124641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-061d5e10-61fc-497f-9e2e-6d927405e7d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:28.139675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:28.140757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:28.141703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:28.144170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 420/830] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:28.145759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.146263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.146737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.166786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:28.168405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:28.208271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e424d-b2ef-4e81-866a-01f69a7a8f5a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:28.241112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:28.251333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 421/831] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:28.251333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:28.258025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c988-5166-4696-a423-9c446128427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:28.261335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c988-5166-4696-a423-9c446128427c None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.261335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c988-5166-4696-a423-9c446128427c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.261335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c988-5166-4696-a423-9c446128427c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.261335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c988-5166-4696-a423-9c446128427c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:28.261904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c988-5166-4696-a423-9c446128427c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:28.283867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:28.338423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 411/832] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:28 2022] POST /identity/v3/endpoints => generated 337 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:28.370122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45cf1f-693f-492f-b852-17533bfd6b68 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:28.446015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:28.446838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.447269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.447641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.448466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:28.448879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:28.450509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:28.470355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 422/833] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:28.476145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c988-5166-4696-a423-9c446128427c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:28.582573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:28.607107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c988-5166-4696-a423-9c446128427c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:28.649976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:28.650602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.650890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.651232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.651946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=81eb5931848749a1ad22d9d228899cda) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:28.652258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:28.654292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:28.662068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 412/834] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:28 2022] GET /identity/v3/role_assignments?user.id=81eb5931848749a1ad22d9d228899cda&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:28.684950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:28.703332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2763455-3f08-4605-9e34-897c72e386ab None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:28.744410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:28.745402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.745864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.746313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.758982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:28.764714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:28.765762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:28.766577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:28.772027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 423/835] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:28.775516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:28.832546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:28.890162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b79d3-c547-485b-8b10-b2cb846e6ad8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:28.926281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:28.927124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:28.927569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:28.927981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:28.928780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:28.929245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:28.930726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:28.942347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 413/836] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:28.945596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:29.053181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c97e5a-7a23-47d9-a257-f8603aabd6ce None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:29.089028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:29.089921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:29.090346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:29.090764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:29.103258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:29.105071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:29.107872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:29.120696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:29.126991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:29.127941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:29.127941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:29.127941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:29.127941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 424/837] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:29.128617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:29.236059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4233d-d41d-4ef2-9266-996c5eba202c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:29.246557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:29.271278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:29.272103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:29.272552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:29.272976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:29.273747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:29.274191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:29.275730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:29.289137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 414/838] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:29.297516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:29.399825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507f2b0-0b0e-4007-8987-a2138c8569b1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:29.434212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:29.434836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:29.435117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:29.435417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:29.443571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:29.445057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:29.456669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:29.457757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:29.458846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 425/839] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:29.463808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:29.568534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26532987-dfb0-4a01-8555-6075ea97145e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:29.603849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:29.604723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:29.605143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:29.605552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:29.606456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:29.606915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:29.608526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:29.627201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 415/840] 10.222.0.62 () {56 vars in 1135 bytes} [Tue Sep 13 09:58:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:29.632578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:29.691485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:29.734884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f8667-4662-4c66-bd35-324ca941e4e4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:29.772072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:29.772695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:29.772975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:29.773245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:29.842773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:29.845130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:29.904826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:29.952110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 426/841] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 09:58:29.958060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:29.987025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:30.061672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ae0c3-ac49-434d-8006-d30973e7c528 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.103913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.103913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.103913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.103913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.115400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.116978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4ba6860e-279c-47d2-b347-bcd9e3a6d02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:30.134641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:30.135665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:30.136539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:30.136539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:30.136539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:30.136539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 416/842] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:30.142232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.231916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61278-d86e-4409-9a27-b155ef4327b1 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.232236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61278-d86e-4409-9a27-b155ef4327b1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.232454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61278-d86e-4409-9a27-b155ef4327b1 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.233827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 417/843] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:30.239144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.239460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.239734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.256020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d36a2-83d4-4dfa-a4ee-9a6ed3654aed None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.259506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:30.260440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:30.292863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.293803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.294181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.294558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.295347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:30.295756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.297262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.309211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 427/844] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:30.316495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.335884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:30.342929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 418/845] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:30.427024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e76ccb-c4eb-4ba4-b1b9-f75c6af4259c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.462913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.463575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.464375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.464375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.471252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.472698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-27c8ca19-4e35-468f-b6fb-2c92d63d8331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:30.484451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:30.485352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:30.486250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 428/846] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:30.491619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.504436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:30.507427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132e440-8615-46e6-bd35-43cf9faa7b0c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.507720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132e440-8615-46e6-bd35-43cf9faa7b0c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.508000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132e440-8615-46e6-bd35-43cf9faa7b0c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.509579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 429/847] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:30.514756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.515548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.515832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.534511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:30.535849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:30.608095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 430/848] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:30.616890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b8d1c-ca8f-4e5d-acc9-6c0542d54fa7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.635563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.644845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:30.668107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.668107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.668107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.668107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.668497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:30.668497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.668497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.678273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 419/849] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:30.684437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.747469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990e4ee-ce5b-4f96-8422-7c45703decc2 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.779649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:30.784343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.784990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.785286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.785542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.786128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.787583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:30.797693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:30.798669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:30.799533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:30.803889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 431/850] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:30.804740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.825088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ce3a2-536f-4a99-8510-c077c8474ac9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.870720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.871639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.872211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.872661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.873818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=174cca9647954ad1afdd4e6b8fba18f4) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:30.875430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.877306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.878027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:30.887732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 420/851] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/role_assignments?user.id=174cca9647954ad1afdd4e6b8fba18f4&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:30.888811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:30.917502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da83528-e776-477f-a9d8-c7c071d50350 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:30.961397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:30.961955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:30.962237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:30.962509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:30.963146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:30.963493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:30.964887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:30.970826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 432/852] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:30.976570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.031765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eabbd4-0072-4ce8-9ac5-20d6a7dac4f2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.079427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.080740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.081031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.081312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.083808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:31.097085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:31.097739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1f210-cb2a-4c80-8dcf-350e6b2a3cf0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:31.103521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:31.104538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 09:58:31.105477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:31.107301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 421/853] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:31.111531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.144620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.144620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.144924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.145160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.145739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:31.147247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.147465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34478b-c080-48aa-bbe7-218e2afdff13 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.149796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:31.152699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 433/854] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:31.178817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f936b03-9d74-4c36-9c4c-a3b4e0c494f5 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.179535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f936b03-9d74-4c36-9c4c-a3b4e0c494f5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.179535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f936b03-9d74-4c36-9c4c-a3b4e0c494f5 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.180919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 434/855] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:31.186743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.187081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.187348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.207280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:31.208672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:31.248057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e331-8661-4a64-b8c2-fde4a05b09af None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.286418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 435/856] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:31.287349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.288165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.288584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.288987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.289794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:31.290487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.292132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.302674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 422/857] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:31.308817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.423046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aa579-b8d5-40c3-8d99-8f4026b25e7f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.451148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:31.460136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.460783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.461068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.461343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.472687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.474715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:31.489357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:31.490319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:31.491616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:31.491616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:31.491616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:31.491616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 436/858] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:31.496587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.563664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:31.567262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.603063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acda11f-70a0-4494-98a8-97dfd6f34c20 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.638924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.640234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.640503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.640780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.641421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:31.641724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.643444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.656985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 423/859] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:31.664650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.691285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb9f45-ddf2-4111-a772-148b931b393c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.741700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.742479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.742763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.743041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.743668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.745927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:31.758389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:31.759461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:31.760329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:31.763348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 437/860] 10.222.0.62 () {56 vars in 1132 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:31.766361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.789076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338713d6-b9ca-4f6b-a4da-f0b7e3e6e182 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.807363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:31.828929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.828929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.829173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:31.829803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.830152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.837651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.839222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:31.852379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:31.853630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:31.854816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:31.861993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 424/861] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:31.863024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.907419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39884a-b524-4a6a-a649-9816cc01b1a5 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:31.957274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:31.958741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:31.959634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:31.960099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:31.961197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:31.961685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:31.963996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:31.970720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 438/862] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:31.976345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:31.989812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5d65-e3c1-4d5e-88a0-819db9eed7e2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:32.033300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:32.033912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:32.034199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:32.034480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:32.035227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:32.035543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:32.037419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:32.059775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 425/863] 10.222.0.62 () {56 vars in 1149 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:32.067636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:32.099422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66508f-122d-4a04-b686-3ddc2305b52a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:32.107032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:32.112202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:32.140477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:32.141661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:32.142341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:32.142929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:32.143881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:32.144405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:32.146237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:32.156877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 439/864] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:32.194561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434cbf-516e-481c-9506-3dbe429a1695 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:32.242121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:32.243312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:32.243822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:32.244315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:32.325698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:32.327238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:32.364766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:32.444508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 426/865] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:58:32.449806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:32.557621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5b242-8435-4b26-a77d-db5586874fe0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:32.592449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:32.593617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:32.593902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:32.594172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:32.605031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:32.606462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:32.621681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:32.622616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:32.623542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:32.623542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:32.623542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:32.623542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 440/866] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:32.631890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:32.737587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409fa2f-7b42-456b-abbc-d0fe2cd3273e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:32.772047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:32.772650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:32.772926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:32.773196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:32.773837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:32.774118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:32.775491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:32.777072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:32.788959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 427/867] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:32.799545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:32.834832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:32.904339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83b26-3942-4b12-a67c-63895083fb75 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:32.940258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:32.941271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:32.941761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:32.942265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:32.949687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:32.951456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:32.964092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:32.965026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:32.965889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:32.968836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 441/868] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:32.974720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:33.081231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04e2b1-59c5-4bc7-b6ac-4a814fbe0950 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:33.120188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:33.121214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.121724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.122211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.123190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:33.123731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:33.125484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c27b05-9e75-4850-956a-4b876c828ced None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:33.146764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 428/869] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:33.153186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:33.234882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f826bf-28e9-4d07-bd48-5998c11ce37e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.235220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f826bf-28e9-4d07-bd48-5998c11ce37e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.235548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f826bf-28e9-4d07-bd48-5998c11ce37e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.237173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 429/870] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:33.242327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42c90-f106-459b-a491-8c0510b74a86 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.242636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42c90-f106-459b-a491-8c0510b74a86 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.242885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42c90-f106-459b-a491-8c0510b74a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.255517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:33.264371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-85a42c90-f106-459b-a491-8c0510b74a86 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:33.268867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-85a42c90-f106-459b-a491-8c0510b74a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:33.268984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6c86d-2e16-4c25-9746-4386d218efb2 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:33.302475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:33.303399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.303872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.304324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.305241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=4c062a52bd70482188def01c1ed57ab2) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:33.305729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:33.307429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5692-6a8f-443d-963b-9b7331dacabd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:33.314917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 442/871] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:33 2022] GET /identity/v3/role_assignments?user.id=4c062a52bd70482188def01c1ed57ab2&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:33.330143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:33.353717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 430/872] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:33.479931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a70bd-7737-4b7a-b772-c439dfc80dbc None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.480412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a70bd-7737-4b7a-b772-c439dfc80dbc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.480897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a70bd-7737-4b7a-b772-c439dfc80dbc None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.482646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 443/873] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:33.488461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.488809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.489188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.508397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:33.509617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:33.588655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 431/874] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:33.634441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a70bd-7737-4b7a-b772-c439dfc80dbc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:33.744500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a70bd-7737-4b7a-b772-c439dfc80dbc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:33.779476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:33.780110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.780422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.780700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.781338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:33.782773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:33.792340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:33.793328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:33.794161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 444/875] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:33.801851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:33.881920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:33.901422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:33.914776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c46cd-50cf-4681-a245-a2dba20f594d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:33.941540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:33.956764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:33.958103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:33.958555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:33.959302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:33.960096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:33.960753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:33.963015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:33.970294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 432/876] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:33.978699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:34.016196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1980fb2-dbc7-4d80-8a6c-83d7898b2db3 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:34.058393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d970c-b8ff-4e80-a72a-7e02ecb09354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:34.059381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d970c-b8ff-4e80-a72a-7e02ecb09354 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:34.059973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d970c-b8ff-4e80-a72a-7e02ecb09354 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:34.060463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d970c-b8ff-4e80-a72a-7e02ecb09354 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:34.061377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d970c-b8ff-4e80-a72a-7e02ecb09354 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:34.063109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d970c-b8ff-4e80-a72a-7e02ecb09354 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:34.091257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f25-496d-4058-862b-755cc133fbbd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:34.127183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:34.128017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:34.128284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:34.128564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:34.129195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:34.129490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:34.130863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee66b-4cc4-46b4-a716-60b01d10ba4c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:34.142010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 433/877] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:34.164689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 445/878] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:33 2022] POST /identity/v3/services => generated 251 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:58:34.466442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4653f-e5b8-4f4b-9d2a-eed8dbf3662a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:34.466787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4653f-e5b8-4f4b-9d2a-eed8dbf3662a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:34.466999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4653f-e5b8-4f4b-9d2a-eed8dbf3662a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:34.469195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 434/879] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:34.474596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563b41f-4464-45e3-8754-58c652296e85 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:34.474887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563b41f-4464-45e3-8754-58c652296e85 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:34.475191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563b41f-4464-45e3-8754-58c652296e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:34.493812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6563b41f-4464-45e3-8754-58c652296e85 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:34.495019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6563b41f-4464-45e3-8754-58c652296e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:34.564575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:34.571948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 446/880] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:34.847576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4653f-e5b8-4f4b-9d2a-eed8dbf3662a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:34.894258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:34.953678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4653f-e5b8-4f4b-9d2a-eed8dbf3662a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:34.992410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ccb5c-09c2-461b-9366-6690072cf3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:34.993021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ccb5c-09c2-461b-9366-6690072cf3d1 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:34.993300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ccb5c-09c2-461b-9366-6690072cf3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:34.993573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ccb5c-09c2-461b-9366-6690072cf3d1 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:34.994230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ccb5c-09c2-461b-9366-6690072cf3d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:34.995606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ccb5c-09c2-461b-9366-6690072cf3d1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:35.130870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 435/881] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:34 2022] POST /identity/v3/services => generated 269 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:35.725655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047353a-35bc-4dbb-b692-11dadf480c6a None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:35.726159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047353a-35bc-4dbb-b692-11dadf480c6a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:35.726159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047353a-35bc-4dbb-b692-11dadf480c6a None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:35.727763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 447/882] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:35.732971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:35.733320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:35.734088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:35.752818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:35.754031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:35.835502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 436/883] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:36.105271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047353a-35bc-4dbb-b692-11dadf480c6a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.212135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047353a-35bc-4dbb-b692-11dadf480c6a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:36.251776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9613ad-8c5b-4156-bd48-31222669142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:36.252416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9613ad-8c5b-4156-bd48-31222669142e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.252700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9613ad-8c5b-4156-bd48-31222669142e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.252976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9613ad-8c5b-4156-bd48-31222669142e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.263633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9613ad-8c5b-4156-bd48-31222669142e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-9f9613ad-8c5b-4156-bd48-31222669142e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:36.269195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:36.270176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:36.271494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 448/884] 10.222.0.62 () {56 vars in 1120 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:36.278401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.310806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:36.329897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b3132-bf98-4982-8bd7-69a1cf91b98c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.330256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b3132-bf98-4982-8bd7-69a1cf91b98c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.330450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b3132-bf98-4982-8bd7-69a1cf91b98c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.333107 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 449/885] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:36.338526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.339078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.339594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.359380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:36.361016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:36.397764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354f6af-62ef-4a49-a1cf-602bdb5d0976 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:36.427627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:36.434032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:36.434834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.435326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.435601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.437002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:36.437499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:36.439062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14913f2a-9208-4568-ace0-37eab0341b8c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:36.447339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 450/886] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:36.448230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 437/887] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:36.459274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.490848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd684a-0ad3-4cff-b0df-3844de60395f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.491202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd684a-0ad3-4cff-b0df-3844de60395f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.491437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd684a-0ad3-4cff-b0df-3844de60395f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.492911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 438/888] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:36.497930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.498221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.498498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.518956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:36.521113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:36.563603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137ae8f-226d-4c95-83e7-30d4c81f5864 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:36.598178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 439/889] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:36.601417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:36.602296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.602797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.603299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.614513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:36.616203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:36.631585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:36.632540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:36.633420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:36.633420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:36.633420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:36.635388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 451/890] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:36.640915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.663193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:36.743670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51978bc5-6f19-43d0-880a-cd88a8773550 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:36.745620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.782797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:36.783780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.784270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.784751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.785582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:36.786085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:36.787662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:36.791459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:36.805665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 440/891] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:36.807372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.856565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b384aea-2379-4d29-868d-a1bac316b45e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:36.897496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:36.899078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:36.899724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.900012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.900287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.900955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:36.902324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:36.916656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d64e-b738-494d-97fe-e3a8f5e3de7e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:36.951516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:36.952181 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:36.952460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:36.952734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:36.959649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:36.969762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:36.972942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:36.973900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 441/892] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:36.974877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:36.987718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 452/893] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:36 2022] POST /identity/v3/services => generated 241 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:58:36.987718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.101316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7a9af-696d-499c-be32-9a51c5d5f123 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.144152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.150574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.151579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.152025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.152486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.153232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.154556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab3805-6c6c-460b-aa42-834f8f91849d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.155822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.160199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:37.168308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:37.169381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:37.170221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:37.173930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 442/894] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:37.180489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.198657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.199326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.199632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.200056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.200960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:37.201265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.202955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.222455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 453/895] 10.222.0.62 () {56 vars in 1149 bytes} [Tue Sep 13 09:58:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:37.229745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.305126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733ed41-7894-45f7-b034-4e31ec353a92 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.348692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.349465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.349862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.350151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.350786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:37.351090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.352491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9819c6-6ef9-4fed-90b5-d773a78ca8e8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.353067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d26c-d7d2-418e-8a30-8b98162d887a None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.358688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 443/896] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:37.360037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff93935-2ca4-41ea-9229-11222f5f2ed8 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.360322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff93935-2ca4-41ea-9229-11222f5f2ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.360594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff93935-2ca4-41ea-9229-11222f5f2ed8 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.362377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 444/897] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:37.364556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff93935-2ca4-41ea-9229-11222f5f2ed8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.399418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.400064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.400346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.400672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.425407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.435776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.470945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.472584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.477177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff93935-2ca4-41ea-9229-11222f5f2ed8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.516394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.517118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.517406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.517680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.518331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:37.518636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.520039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5910f-db61-4e51-b4c2-9c9d8bfcde0c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.525774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 445/898] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:37.526789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.527078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.527394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.550152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:37.551726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:37.586466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 454/899] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 09:58:37.587688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.635217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 446/900] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:37.635217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.652803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.697568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.720111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df21852-6828-4359-a044-6f9087242bbd None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.744679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb0309-3a20-445f-8fd1-fe4319006f04 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.766941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.768665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.769365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.769797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.770904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] RBAC: Adding query filter params (interface=public, service_id=cad4f106c7714bc7b434e98000056ed8, region_id=RegionOne) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:37.771799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.773825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.782724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.783527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.783882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.784229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.785185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 455/901] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/endpoints?service_id=cad4f106c7714bc7b434e98000056ed8&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:37.790679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.799056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.800542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:37.819620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:37.820631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:37.821521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:37.821521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:37.821521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:37.821521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 447/902] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:37.826737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.915728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad332-b1cc-42b6-9d98-d77bf2c3e541 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.934038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.947902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8e59-915e-4544-9e0b-298ca960de55 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:37.952563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.966305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.966305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.966305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.966305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.966305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:37.966789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:37.966878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 456/903] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 09:58:37.966878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:37.991786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:37.992622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:37.993059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:37.993459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:37.994276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:37.994679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:37.996731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.011407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 448/904] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:38.017135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.103424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f2424-34ac-4fff-be9a-e843cce0be38 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.137884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d02f1-bdf1-457d-ab46-a7f0fcc7ac97 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.148248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.148774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.149396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.149690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.151609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.152125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:38.161099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:38.162095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:38.162967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 457/905] 10.222.0.62 () {56 vars in 1132 bytes} [Tue Sep 13 09:58:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:38.169273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.183636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.185432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.185432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.185713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.193942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.195820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:38.210363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:38.211568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:38.212492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 449/906] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:38.217396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.241090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:38.261020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:38.315432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909588-5f89-450b-8042-cfbffe55204b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.349638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd08a9a-72e9-4a1b-8b9c-5de644af129e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.358897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.359568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.360280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.360555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.361188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:38.361495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.363245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.371214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 458/907] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:38.375769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.403137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.403780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.404061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.404334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.405611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:38.405919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.407824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.427376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 450/908] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:38.429525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.510483 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0c4fc-e99d-40e4-8e4a-9ce005093fd9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.522386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:38.537440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:38.548307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71432ed9-b713-44ba-8aab-69890be2b1d9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.559671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.565267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.565267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.565267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.565267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:38.565267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.565267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.571494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 459/909] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:38.584170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.590972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.591677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.591967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.592305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.593953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] RBAC: Adding query filter params (role.id=a3ba0e9ce6fc4630ac55028e1746b649, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=4c062a52bd70482188def01c1ed57ab2) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:38.594879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.599525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.605672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 451/910] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/role_assignments?user.id=4c062a52bd70482188def01c1ed57ab2&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=a3ba0e9ce6fc4630ac55028e1746b649 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:38.711793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba88b5-bbee-4770-8c57-b5de8576a4dc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.749938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.750564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.750844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.751116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.751858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] RBAC: Adding query filter params (interface=public, service_id=4b7faed89e5647eba14409029b9b8e39, region_id=RegionOne) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:38.752167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.753550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7083f13-f61f-436a-ac0a-6c3dd6adda30 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.759901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 460/911] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/endpoints?service_id=4b7faed89e5647eba14409029b9b8e39&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:38.765324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:38.880025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15175a-67f8-4ae1-8395-bdbdc98bff19 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:38.917136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8103c0b-f112-471e-beef-4e2f6c03e338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:38.918899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8103c0b-f112-471e-beef-4e2f6c03e338 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:38.918899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8103c0b-f112-471e-beef-4e2f6c03e338 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:38.918899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8103c0b-f112-471e-beef-4e2f6c03e338 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:38.919317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8103c0b-f112-471e-beef-4e2f6c03e338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:38.921016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8103c0b-f112-471e-beef-4e2f6c03e338 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:38.929367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 452/912] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:38 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 09:58:39.250818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1ee9a-a809-40b5-afbc-79a468ac61b4 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:39.251718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1ee9a-a809-40b5-afbc-79a468ac61b4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:39.252137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1ee9a-a809-40b5-afbc-79a468ac61b4 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:39.254176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 461/913] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:39.261377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:39.261941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:39.262427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:39.281995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:39.283560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:39.298620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:39.356030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 453/914] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:39.623317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1ee9a-a809-40b5-afbc-79a468ac61b4 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:39.664900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:39.730955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1ee9a-a809-40b5-afbc-79a468ac61b4 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:39.766217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:39.766845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:39.767122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:39.767417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:39.767990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:39.769346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:39.778126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:39.779102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:39.779998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 462/915] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:39.785280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:39.897917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663a38-78d1-4f5c-969c-2316f49b29ad None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:39.939194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:39.939194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:39.939194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:39.939194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:39.939615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:39.939615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:39.939615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:39.943829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 454/916] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:39.955195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:40.075059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25741-c6d9-416e-8e2d-060d5570247d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:40.076601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:40.111407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:40.112011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.112370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.112690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.113314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:40.113662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:40.115182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:40.120524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 463/917] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:40.126621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:40.142658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:40.239297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dbd59-a35d-405a-80f9-15f3e15d81ec None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:40.286530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:40.287191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.287480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.287758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.288477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] RBAC: Adding query filter params (interface=public, service_id=c98fc32fc0dc46f1abba67c9dfad8650, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:40.288858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:40.290560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0921aa0-f5eb-46b1-a2a9-3e1903d9bef6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:40.297558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 455/918] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:40 2022] GET /identity/v3/endpoints?service_id=c98fc32fc0dc46f1abba67c9dfad8650&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:40.303339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:40.401270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e819-524d-4562-b223-6c8a5387ed51 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.401762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e819-524d-4562-b223-6c8a5387ed51 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.401842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e819-524d-4562-b223-6c8a5387ed51 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.403352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 456/919] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:40.411205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54707-bee0-4099-9c06-3e32dd4ba6b9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.411205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54707-bee0-4099-9c06-3e32dd4ba6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.411205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54707-bee0-4099-9c06-3e32dd4ba6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.415417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabe935-1af6-4243-a39b-772ec209c246 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:40.434509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1cd54707-bee0-4099-9c06-3e32dd4ba6b9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:40.437991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1cd54707-bee0-4099-9c06-3e32dd4ba6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:40.481648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65481d6-3002-4d27-a629-41530980592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:40.482779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65481d6-3002-4d27-a629-41530980592a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.483345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65481d6-3002-4d27-a629-41530980592a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.483823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65481d6-3002-4d27-a629-41530980592a None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.484626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65481d6-3002-4d27-a629-41530980592a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:40.486217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65481d6-3002-4d27-a629-41530980592a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:40.493108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 464/920] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:40 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 09:58:40.568411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 457/921] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:40.845916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a1d8-6df9-4e0a-a847-050507797d8a None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.846102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a1d8-6df9-4e0a-a847-050507797d8a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.846378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a1d8-6df9-4e0a-a847-050507797d8a None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.847979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 465/922] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:40.853262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:40.853552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:40.853824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:40.867403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:40.874158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:40.875689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:40.896252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820a1d8-6df9-4e0a-a847-050507797d8a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:40.929260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:40.956997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 458/923] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:41.009045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820a1d8-6df9-4e0a-a847-050507797d8a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.046103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.046740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.047071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.047375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.047960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.049369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:41.058747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:41.059799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:41.060710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 466/924] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:41.066615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.180123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a4b3-f435-4f9d-bdec-f18a0c8701a3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.215958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496373f-9471-4496-8957-115cead68439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.216912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496373f-9471-4496-8957-115cead68439 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.217458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496373f-9471-4496-8957-115cead68439 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.217934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496373f-9471-4496-8957-115cead68439 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.218765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496373f-9471-4496-8957-115cead68439 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:41.219319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496373f-9471-4496-8957-115cead68439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.220896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496373f-9471-4496-8957-115cead68439 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.223205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:41.224232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.227812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 459/925] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:41.234974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496373f-9471-4496-8957-115cead68439 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.329049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:41.335657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8acf3-a0e3-450e-80b4-41edb24102d7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.348109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496373f-9471-4496-8957-115cead68439 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.370778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869503d-f21f-4e51-9608-b3be3b3f9758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.371452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869503d-f21f-4e51-9608-b3be3b3f9758 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.371734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869503d-f21f-4e51-9608-b3be3b3f9758 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.372007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869503d-f21f-4e51-9608-b3be3b3f9758 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.388145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869503d-f21f-4e51-9608-b3be3b3f9758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-1869503d-f21f-4e51-9608-b3be3b3f9758 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:41.393505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:41.394393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:41.395232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 467/926] 10.222.0.62 () {56 vars in 1120 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:41.396488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765848-3a89-4e2a-bb57-442ef69af330 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.396488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765848-3a89-4e2a-bb57-442ef69af330 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.396488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765848-3a89-4e2a-bb57-442ef69af330 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.397695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 468/927] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:41.400186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.400807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.401090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.401591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.402107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:41.402342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10765848-3a89-4e2a-bb57-442ef69af330 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.402441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.403738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582abe7-672d-4ddc-b8de-dde8572bf39e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.409246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 460/928] 10.222.0.62 () {56 vars in 1114 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:41.410402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.410685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.410957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.429148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:41.430389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:41.493349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:41.511191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 461/929] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:41.511191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.538060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10765848-3a89-4e2a-bb57-442ef69af330 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.584615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.585383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.585896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.586360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.587260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:41.587770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.589491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.599409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 469/930] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:41.604682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.623860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85104b8-c3ac-4472-8ea3-53e7e21119ee None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.628160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:41.665830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.666787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.667304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.667776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.668658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.670276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.718255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc4330-736b-49b6-868e-d972f10e06f6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.755730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 462/931] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:41.766174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.766816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.767091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.767405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.780817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.782399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.788122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:41.803738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:41.804752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:41.805655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:41.805655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:41.805655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:41.805655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 470/932] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:41.812271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.855422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:41.884999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:41.917126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecb505-b5ef-4d03-8065-db9501867e31 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.931219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067145f-7038-4983-8e97-6fd5ddb21f9c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:41.958162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.958394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.958625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.958797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.959434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.961633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:41.976166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:41.977105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:41.978613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 463/933] 10.222.0.62 () {56 vars in 1132 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:41.984242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:41.984437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:41.984437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:41.984836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:41.984836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:41.985463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:41.985965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:41.988316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.002900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 471/934] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:42.008827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.114742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff4919-8c87-4485-af07-fa4f55cac83a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.130747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94636a4-63be-49e8-9cc9-b89eea2df674 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.132860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:42.160678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.161344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.162302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:42.162414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.163338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.164891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:42.165793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.167475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.175946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.176872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.181850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.181850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.182013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 464/935] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:42.182013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.186619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.188354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:42.203345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:42.204316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:42.205262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:42.208304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 472/936] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:42.214874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.300012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfb6d3-52a1-48ff-9365-4c2391e60149 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.323634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ed5b5-94e6-4402-ac5b-b5a78de631c1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.348136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.348903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.349198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.349471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.350106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:42.350706 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.351927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.361949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.363154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 465/937] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:42.363537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.364014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.364119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.364629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.365576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:42.366085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.367763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.403250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 473/938] 10.222.0.62 () {56 vars in 1149 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:42.406220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.429905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:42.456496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:42.495756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf2bfb-9373-420e-bfde-72d7527e0057 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.530530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35951f2-3193-47f0-82d9-608f0173537c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.533648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d661bca-3ac8-4efa-9a0b-8bb05ca5f06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.534524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661bca-3ac8-4efa-9a0b-8bb05ca5f06f None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.535004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661bca-3ac8-4efa-9a0b-8bb05ca5f06f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.535508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661bca-3ac8-4efa-9a0b-8bb05ca5f06f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.536386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d661bca-3ac8-4efa-9a0b-8bb05ca5f06f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.537953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d661bca-3ac8-4efa-9a0b-8bb05ca5f06f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.568169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.568759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.569094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.569377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.580563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.582045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:42.598244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:42.599327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:42.600324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:42.600324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:42.600324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:42.600324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 474/939] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:42.606341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.646128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 466/940] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:42.720735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:42.728946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f646a6-ddd4-4905-a8eb-473d334176c1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.775845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.776481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.776761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.777037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.777720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:42.778025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.779536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.797948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 475/941] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:42.799895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222c39-0919-49bb-abea-ddbf62f19576 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.800111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222c39-0919-49bb-abea-ddbf62f19576 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.800348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222c39-0919-49bb-abea-ddbf62f19576 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.800949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:42.802002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 467/942] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:42.807654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.807947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.808220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.830077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:42.831243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:42.927421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 468/943] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:42.928036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705ce07-5edb-4a48-a8dd-425208eff59c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:42.966150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:42.966781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:42.967064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:42.967888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:42.974871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:42.976403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:42.988293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:42.989158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:42.990034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:42.994082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 476/944] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:42.995750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:43.024231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:43.101742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4406c95-e894-451f-8300-5959c4471e90 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:43.137385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:43.138047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:43.138325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:43.138647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:43.139411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:43.139718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:43.141138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:43.158682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 469/945] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:43.160045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:43.173632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:43.236929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:43.273457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430505f-a1fb-4a6a-8b5d-3951a6569af7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:43.317612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:43.318283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:43.318569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:43.318847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/4c062a52bd70482188def01c1ed57ab2/roles/a3ba0e9ce6fc4630ac55028e1746b649` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:43.321167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=4c062a52bd70482188def01c1ed57ab2, role_id=a3ba0e9ce6fc4630ac55028e1746b649)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:43.323382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:43.345080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:43.357647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1064a-b2e7-44ce-aa9a-d0b7132f3031 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:43.407224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 477/946] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:58:43 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/4c062a52bd70482188def01c1ed57ab2/roles/a3ba0e9ce6fc4630ac55028e1746b649 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 09:58:43.409486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:43.410450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:43.410926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:43.411450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:43.412242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:43.413987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:43.425456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:43.426562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:43.427445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:43.430205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 470/947] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:43.437665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:43.520881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:43.558008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb782e6-5261-458c-b4d9-5a9ea07c2497 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:43.599186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:43.599820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:43.600109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:43.600384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:43.601025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:43.601327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:43.602728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:43.609029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 478/948] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:43.617063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:43.728549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b1e1-f35e-4d55-9db2-d42fac1768a0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:43.762908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:43.763568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:43.763992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:43.764325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:43.764942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:43.765246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:43.766610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681e5e-661c-4200-a981-99f54bae0ef8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:43.772003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 471/949] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:43.777858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:43.886069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c47a0-3c79-46c4-9a76-387ceadeb091 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:43.922287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27a56f-1e63-43da-85bb-73d919a72d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:43.922895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27a56f-1e63-43da-85bb-73d919a72d9e None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:43.923188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27a56f-1e63-43da-85bb-73d919a72d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:43.923477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27a56f-1e63-43da-85bb-73d919a72d9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:43.924145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27a56f-1e63-43da-85bb-73d919a72d9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:43.925499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27a56f-1e63-43da-85bb-73d919a72d9e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:43.930362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:43.997586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 479/950] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:44.162131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192ba9-4905-465f-9cd5-aa85a665641b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:44.162379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192ba9-4905-465f-9cd5-aa85a665641b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:44.162809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192ba9-4905-465f-9cd5-aa85a665641b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:44.164613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 472/951] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:44.171184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:44.171184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:44.171436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:44.190321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:44.191614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:44.271190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 480/952] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:44.545031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23192ba9-4905-465f-9cd5-aa85a665641b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:44.564685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:44.655642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23192ba9-4905-465f-9cd5-aa85a665641b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:44.691816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:44.692471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:44.692759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:44.693035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:44.693683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:44.695110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:44.697793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING keystone.server.flask.common [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 09:58:44.746213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.sql.core [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 09:58:44.746213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 09:58:44.746213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [parameters: {'id': '044399852c2f4392a5ca158210c3ff04', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 09:58:44.746213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81737) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 044399852c2f4392a5ca158210c3ff04.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 044399852c2f4392a5ca158210c3ff04. Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application conn.query(q) Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 09:58:44.754105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result.read() Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 09:58:44.754915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application next(self.gen) Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application yield resource Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application next(self.gen) Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise self.value Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application session.commit() Sep 13 09:58:44.755900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.session.flush() Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application compat.raise_( Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 09:58:44.756778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 09:58:44.757681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application conn.query(q) Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result.read() Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 09:58:44.758487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [parameters: {'id': '044399852c2f4392a5ca158210c3ff04', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:44.759518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:44.760373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 044399852c2f4392a5ca158210c3ff04. Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:44.761336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 473/953] 10.222.0.62 () {60 vars in 1140 bytes} [Tue Sep 13 09:58:44 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:44.761906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:44.875860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a717b5-f65d-4162-be82-13d92ad1663e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:44.893519 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:44.913530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:44.915683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:44.915683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:44.915683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:44.921886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:44.923410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-30b2f059-7a61-401e-97a9-e836b018b96a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:44.934804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:44.937074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:44.940827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 481/954] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:44.947196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:44.961316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968189-92f1-4bae-9ccc-863abbb98084 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:44.961316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968189-92f1-4bae-9ccc-863abbb98084 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:44.961316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968189-92f1-4bae-9ccc-863abbb98084 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:44.962185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 482/955] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:44.968830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:44.969346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:44.969771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:44.989365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:44.990956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:45.033390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:45.072646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f61f-4f09-4059-b286-a7c3e7ce6975 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:45.079020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 483/956] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:45.114236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:45.114853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:45.115134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:45.115467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:45.116127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:45.116432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:45.118064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:45.139643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 474/957] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:45.351028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:45.466712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa93d0d-c4ae-45bd-b252-2044f5991ea8 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:45.474776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:45.505395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:45.506563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:45.507052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:45.507557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:45.508373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:45.515150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-2fc3e07a-0380-4515-b3dd-fb660f7ade2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:45.526167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:45.527756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:45.528637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:45.532711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 484/958] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:45.539328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:45.630964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0da368-84a4-4750-98ab-7dcc7dc8a823 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:45.631354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0da368-84a4-4750-98ab-7dcc7dc8a823 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:45.631541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0da368-84a4-4750-98ab-7dcc7dc8a823 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:45.633160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 485/959] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:45.639530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:45.640630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:45.642037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:45.647355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eec807-efcb-42fd-a4b7-42a8c70736f1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:45.671148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:45.672909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:45.688101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:45.694859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:45.695889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:45.696198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:45.696476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:45.697123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:45.697433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:45.699119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:45.706127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 475/960] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:45.713217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:45.791192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 486/961] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:45.850208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ad3d-1185-492a-b64b-e6b676dd3b6c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:45.890354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:45.890973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:45.891289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:45.891568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:45.892204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:45.892509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:45.893882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:45.899457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 476/962] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:46.077528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.194001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:46.197738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be0c28-3baf-423f-9773-55b803f2f061 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:46.249903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b28d-f1d0-43b3-a595-b6572bf4dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:46.250917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b28d-f1d0-43b3-a595-b6572bf4dc48 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.251441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b28d-f1d0-43b3-a595-b6572bf4dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.251964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b28d-f1d0-43b3-a595-b6572bf4dc48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.263042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b28d-f1d0-43b3-a595-b6572bf4dc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-9b90b28d-f1d0-43b3-a595-b6572bf4dc48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:46.268104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:46.269045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:46.269864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:46.273578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 487/963] 10.222.0.62 () {56 vars in 1120 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:46.280234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.337308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f1c2-64c1-459a-bcf9-1d9c54daed02 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.337649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f1c2-64c1-459a-bcf9-1d9c54daed02 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.337872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f1c2-64c1-459a-bcf9-1d9c54daed02 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.339494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 488/964] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:46.347205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.347205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.347205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.353737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:46.365740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:46.367341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:46.400549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed0356-6e78-46c6-bf7f-2bb3a9a4ecc2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:46.438100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:46.438725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.439066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.439375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.440088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:46.440403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:46.441895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:46.453093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 477/965] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:46.457816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.459922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 489/966] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:46.565409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959fe21-3f38-4f55-818b-9ac21f48123e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:46.588483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:46.601193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:46.607851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.607851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.607851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.614148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:46.615715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:46.631595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:46.632545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:46.633429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:46.633429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:46.633429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:46.636507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 478/967] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:46.642482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.736629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:46.746479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.747135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b48d5-53aa-40f4-bc82-83825a24538f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:46.791864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:46.792492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.792765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.793000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.793638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:46.793938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:46.795411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:46.811850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 490/968] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:46.814635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.874115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b8b5d-d78c-4a16-8900-6d755b41164a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:46.918392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:46.919025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:46.919332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:46.919600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:46.920177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:46.921914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:46.931716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:46.932839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:46.933670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:46.938522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 479/969] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:46.939791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:46.959065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117c5fa-859e-4f5d-9616-4463c0325e35 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:46.986527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.003003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.003003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.003003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.003003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.011044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.012521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:47.024986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:47.025952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:47.026875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 491/970] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:47.034283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.052130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.065291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340c14-44b3-431f-aa69-1af5f2973fac None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.110717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.111535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.111878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.112547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.113197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:47.113498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.115370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.125947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 480/971] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:47.128159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.161212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa606b-1568-4c15-b58a-aa205aded384 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.200434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.201209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.201518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.201792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.202528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:47.202832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.204608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.230798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 492/972] 10.222.0.62 () {56 vars in 1149 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:47.232467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.258137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af71611-f10e-4567-8909-12c8133cd543 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.259862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.305600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.306555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.307294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.307604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.308951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:47.309259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.310609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.316551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 481/973] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:47.317087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.323295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.363422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33a8b8-ea6a-4d01-be51-04d60abdc87c None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.425797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0471-2dea-4034-9d41-0a78a2a753f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.426436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0471-2dea-4034-9d41-0a78a2a753f0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.426714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0471-2dea-4034-9d41-0a78a2a753f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.426983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0471-2dea-4034-9d41-0a78a2a753f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.449009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4089e-93b3-4aaf-962b-530c1999979e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.488091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.489014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.489014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.489277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.489924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] RBAC: Adding query filter params (interface=admin, service_id=c98fc32fc0dc46f1abba67c9dfad8650, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:47.490235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.491619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8637-d3cc-4289-b921-a4426f3dc568 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.501150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 482/974] 10.222.0.62 () {56 vars in 1249 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/endpoints?service_id=c98fc32fc0dc46f1abba67c9dfad8650&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:47.501150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1fc7f-32c6-4d6c-8201-da332af2233d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.501150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1fc7f-32c6-4d6c-8201-da332af2233d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.501150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1fc7f-32c6-4d6c-8201-da332af2233d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.501150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 483/975] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:47.505048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1fc7f-32c6-4d6c-8201-da332af2233d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.512272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d0471-2dea-4034-9d41-0a78a2a753f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.514064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d0471-2dea-4034-9d41-0a78a2a753f0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.538990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.581246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.624139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 493/976] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 09:58:47.624330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1fc7f-32c6-4d6c-8201-da332af2233d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.624414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.624414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.624414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.640018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:47.641238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:47.662203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.663087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.663536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.664002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.664888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.666640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.673510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 484/977] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 09:58:47.674311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.739195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 494/978] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:47.794548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.803558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c474f5-54ef-4312-b7dd-0fe06479a027 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:47.843111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:47.843805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:47.844084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:47.844353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:47.855835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:47.857339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:47.872244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:47.873208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:47.874159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:47.874159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:47.874159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:47.874159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 485/979] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:47.879047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:47.980694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:47.985022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8df6-a39c-44d2-b4be-3ac79131d607 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.020348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.020966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.021249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.021523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.022167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:48.022470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.023935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.036369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 495/980] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:48.042457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.081790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.168334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876777d-73a9-4052-95c5-2eef817421ca None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.178851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fa8-ab3d-4b48-bb43-556979edbef6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.208859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.209500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.209781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.210056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.216722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.218193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.218599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.219439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.220050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.220767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.221577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.223248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.226271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING keystone.server.flask.common [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-30df18d9-740a-45b5-9630-57de33324030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:58:48.229512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:48.230368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:48.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 486/981] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:48.231791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6a7b-4ba5-4004-9f81-4ebd47439ebf None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.232153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6a7b-4ba5-4004-9f81-4ebd47439ebf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.232299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6a7b-4ba5-4004-9f81-4ebd47439ebf None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.237687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 487/982] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:48.237687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6a7b-4ba5-4004-9f81-4ebd47439ebf None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.246643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:48.277222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.sql.core [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 09:58:48.277222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 09:58:48.277222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [parameters: {'user_id': 'a344629a3b024389a0c2b33da6d00bea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 09:58:48.277222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81736) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-85dcd670-d640-4eda-8656-05a3a06cf153 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application conn.query(q) Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application result.read() Sep 13 09:58:48.294316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 09:58:48.295461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application next(self.gen) Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application yield resource Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application next(self.gen) Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise self.value Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application session.commit() Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 09:58:48.296431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self.session.flush() Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application compat.raise_( Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 09:58:48.297334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 09:58:48.298330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application conn.query(q) Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application result.read() Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 09:58:48.300252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a344629a3b024389a0c2b33da6d00bea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:48.302738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 09:58:48.305453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 496/983] 10.222.0.62 () {60 vars in 1131 bytes} [Tue Sep 13 09:58:48 2022] POST /identity/v3/users => generated 160 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.308337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.319957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:48.321598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:48.337317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:48.348846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6a7b-4ba5-4004-9f81-4ebd47439ebf None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.387963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.388634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.388931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.389776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.390927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:48.390927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.392304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.407746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 497/985] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:48.408060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 488/985] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:48.409209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.410777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.513729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:48.521068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c238131-4ce3-4481-9201-7b3cc2c5d171 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.548472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3cde0-0b81-4bfc-bd1c-8234cba0d1e9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.562565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.563456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.563922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.564394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.572091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.574002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:48.586977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:48.588093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 09:58:48.589094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:48.595243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 498/986] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:48.596097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.599955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.600587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.600870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.601143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.603032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] RBAC: Adding query filter params (role.id=a3ba0e9ce6fc4630ac55028e1746b649, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=4c062a52bd70482188def01c1ed57ab2) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:48.603375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.604818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.613478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:48.615300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 489/987] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:58:48 2022] GET /identity/v3/role_assignments?user.id=4c062a52bd70482188def01c1ed57ab2&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=a3ba0e9ce6fc4630ac55028e1746b649 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:48.683355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:48.744568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156eba9-79e5-4ff0-9a4e-2820679983ac None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.799202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.799202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.799202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.799202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.799627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:48.799627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.799627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ddd16-489b-4182-a3d5-11c809d51e63 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.806334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07988132-d0f2-4389-95b8-8727b1c7bc98 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:48.838726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 499/988] 10.222.0.62 () {56 vars in 1101 bytes} [Tue Sep 13 09:58:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:48.838911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:48.853981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae2550-cd50-423b-a44b-619b4e0a94c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:48.858363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2550-cd50-423b-a44b-619b4e0a94c8 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:48.858363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2550-cd50-423b-a44b-619b4e0a94c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:48.858363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2550-cd50-423b-a44b-619b4e0a94c8 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:48.858363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2550-cd50-423b-a44b-619b4e0a94c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:48.858363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2550-cd50-423b-a44b-619b4e0a94c8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:48.933443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 490/989] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:48 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:58:49.905123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745899a-2858-4523-8087-34298a8a32ef None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:49.905651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745899a-2858-4523-8087-34298a8a32ef None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:49.905651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745899a-2858-4523-8087-34298a8a32ef None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:49.907063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 500/990] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:49.912225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:49.912620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:49.912796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:49.931633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:49.933157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:50.007307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 491/991] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:50.278786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745899a-2858-4523-8087-34298a8a32ef None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:50.349176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:50.392183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745899a-2858-4523-8087-34298a8a32ef None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:50.428418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:50.429042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:50.429327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:50.429602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:50.430179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:50.431578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:50.441246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:50.442214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 09:58:50.443065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:50.447333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 501/992] 10.222.0.62 () {56 vars in 1126 bytes} [Tue Sep 13 09:58:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:50.448464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:50.550214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc8c3a-69ce-49f9-9248-9e0977d187f5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:50.566960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:50.583951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:50.584559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:50.584832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:50.585097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:50.585716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:50.586014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:50.587392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:50.597364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 492/993] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:50.598470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:50.703751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b788dd2-8595-4bae-bc9d-1f3ceb566227 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:50.734322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:50.740122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:50.740754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:50.741036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:50.741302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:50.741967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:50.742359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:50.743889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a1cf4-e378-4273-9503-ee5ed0d74ac7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:50.754749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 502/994] 10.222.0.62 () {56 vars in 1122 bytes} [Tue Sep 13 09:58:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:50.756272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:50.860928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483c386-304a-4a67-97c5-fed96f488dd8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:50.861550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9305e1-0d22-4e87-84e5-71d084e1a45b None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:50.862071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9305e1-0d22-4e87-84e5-71d084e1a45b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:50.862603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9305e1-0d22-4e87-84e5-71d084e1a45b None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:50.866271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 503/995] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:50.870304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e9e12-3725-4dd3-8a5e-2ac9d34ab738 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:50.870811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e9e12-3725-4dd3-8a5e-2ac9d34ab738 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:50.871094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e9e12-3725-4dd3-8a5e-2ac9d34ab738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:50.890094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-037e9e12-3725-4dd3-8a5e-2ac9d34ab738 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:50.891510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-037e9e12-3725-4dd3-8a5e-2ac9d34ab738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:50.897257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28bdd0-9706-4fb8-9ea2-d7d41428fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:50.897866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28bdd0-9706-4fb8-9ea2-d7d41428fec8 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:50.898145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28bdd0-9706-4fb8-9ea2-d7d41428fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:50.898414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28bdd0-9706-4fb8-9ea2-d7d41428fec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:50.899072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28bdd0-9706-4fb8-9ea2-d7d41428fec8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:50.900482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28bdd0-9706-4fb8-9ea2-d7d41428fec8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:50.959515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 504/996] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:50.974564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 493/997] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:51.130993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e797-cfe3-463a-bc1b-8069e0802e16 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.131365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e797-cfe3-463a-bc1b-8069e0802e16 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.131491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e797-cfe3-463a-bc1b-8069e0802e16 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.133061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 505/998] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:51.138448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.138741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.139581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.208295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:51.209551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:51.228619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e797-cfe3-463a-bc1b-8069e0802e16 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:51.246370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:51.286895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 494/999] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:51.336370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e797-cfe3-463a-bc1b-8069e0802e16 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:51.361719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:51.370849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:51.371512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.371796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.372066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] PATH_INFO: `/v3/services/image` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.376567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:51.376567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:51.387273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:51.388269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:58:51.389704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:51.390521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 506/1000] 10.222.0.62 () {56 vars in 1105 bytes} [Tue Sep 13 09:58:51 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:51.391483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:51.496382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21b6f-bbfc-44d3-902f-cb0e51e857f1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:51.531474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:51.532355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.532787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.533241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.534018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] RBAC: Adding query filter params (name=image) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:51.534469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:51.536009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:51.541457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 495/1001] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:51.546781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:51.555111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:51.586862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:51.651293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244acf98-5921-47f8-9a3c-392a8a5c4461 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:51.660916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668506-9b8a-4393-84ce-6e0e6f6bc186 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:51.689835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:51.690446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.690728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.691004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.691661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] RBAC: Adding query filter params (type=image) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:51.691968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:51.693337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:51.698229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 507/1002] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:51.708134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:51.708814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.709097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.709372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.709973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:51.711440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:51.721658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:51.722727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:51.727850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 496/1003] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:58:51.729819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:51.792535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:51.841901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658db41-7c28-466a-b09a-703eb8d7e900 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:51.878965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:51.879648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:51.879920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:51.880192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:51.880839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:51.881144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:51.882561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:51.888316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 508/1004] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:51.894089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:51.990288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:51.997734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11ccc8-fea8-4c66-a1f3-4ea9ea6a0093 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:52.030614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:52.031283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:52.031618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:52.032002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:52.032978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:52.033310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:52.034666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:52.044521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 497/1005] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:52.046752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:52.149069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0c5e7-b114-4e72-812c-f1272bcc402b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:52.166734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:52.184285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:52.184901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:52.185181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:52.185456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:52.186190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] RBAC: Adding query filter params (interface=public, service_id=64aed4976e374d168fe1c2c472e8fbeb, region_id=RegionOne) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:52.186501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:52.187902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9a8a-6da1-47b9-adf2-8a133faaa711 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:52.193142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 509/1006] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:52 2022] GET /identity/v3/endpoints?service_id=64aed4976e374d168fe1c2c472e8fbeb&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:52.198554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:52.300269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de088-de81-4db3-a0ad-a436a425a492 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:52.333835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7681466-66f0-438f-a6ec-ad1eab1a0920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:52.334443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7681466-66f0-438f-a6ec-ad1eab1a0920 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:52.334728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7681466-66f0-438f-a6ec-ad1eab1a0920 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:52.335000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7681466-66f0-438f-a6ec-ad1eab1a0920 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:52.335624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7681466-66f0-438f-a6ec-ad1eab1a0920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:52.336980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7681466-66f0-438f-a6ec-ad1eab1a0920 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:52.351253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 498/1007] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:52 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 09:58:53.168300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df03fb-1467-4d71-900b-f7858040b868 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:53.168846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df03fb-1467-4d71-900b-f7858040b868 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:53.168846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df03fb-1467-4d71-900b-f7858040b868 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:53.170773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 510/1008] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:53.175936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:53.176343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:53.176503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:53.195023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:53.196248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:53.213673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:53.266152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 499/1009] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:53.535371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df03fb-1467-4d71-900b-f7858040b868 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:53.644972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df03fb-1467-4d71-900b-f7858040b868 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:53.680125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadf64a-0a94-49f8-937b-9501745f898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:53.680759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadf64a-0a94-49f8-937b-9501745f898a None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:53.681040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadf64a-0a94-49f8-937b-9501745f898a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:53.681313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadf64a-0a94-49f8-937b-9501745f898a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:53.682432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadf64a-0a94-49f8-937b-9501745f898a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:53.683884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadf64a-0a94-49f8-937b-9501745f898a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:53.686135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 511/1010] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:53.691697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:53.793848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810d794-d1db-40e9-9dac-4820d7f2dd9e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:53.828252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:53.828858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:53.829139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:53.829412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:53.830074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:53.831525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:53.880260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b73d0-9f8d-480a-8142-47b00988f536 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:53.880545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b73d0-9f8d-480a-8142-47b00988f536 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:53.880819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b73d0-9f8d-480a-8142-47b00988f536 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:53.882172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 512/1011] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:53.887017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ba1c6-39d3-4a78-8380-c69fcffa777d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:53.887338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ba1c6-39d3-4a78-8380-c69fcffa777d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:53.887607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ba1c6-39d3-4a78-8380-c69fcffa777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:53.905657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6f4ba1c6-39d3-4a78-8380-c69fcffa777d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:53.906842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6f4ba1c6-39d3-4a78-8380-c69fcffa777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:53.951211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 500/1012] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:53 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:53.968457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:53.993888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 513/1013] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:54.270059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:54.274043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:54.379218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebec2e-37aa-42b7-b793-0e97ddeb72a0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:54.422651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:54.423348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:54.423627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:54.423892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:54.424545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:54.425896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:54.495528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938f58-978a-4e96-9826-c4781184503d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:54.496079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938f58-978a-4e96-9826-c4781184503d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:54.496079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938f58-978a-4e96-9826-c4781184503d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:54.497411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 514/1014] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:54.502309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:54.502593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:54.502871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:54.521314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:54.522434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:54.527203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 501/1015] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:58:54 2022] POST /identity/v3/services => generated 242 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:54.608116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 515/1016] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:54 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:54.880440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:54.944905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:54.983756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eeabb-b0b9-45e4-99ce-b764280d4a76 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:55.018549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:55.019227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:55.019514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:55.019789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:55.020374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:55.021767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:55.031558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:55.032532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:55.033484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 502/1017] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:55.040785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:55.142909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0ac47-2e41-49ea-849b-79efbdde0794 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:55.150131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:55.178094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:55.178705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:55.178987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:55.179288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:55.179915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:55.180217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:55.181575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:55.186495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 516/1018] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:55.192121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:55.293877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681949d-671d-458d-ade4-17fcc119dec3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:55.320326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:55.329274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:55.341346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:55.341346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:55.341346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:55.341346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:55.341346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:55.341346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52c795-34b6-40e1-8476-61cb11062665 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:55.341870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 503/1019] 10.222.0.62 () {56 vars in 1112 bytes} [Tue Sep 13 09:58:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:55.343728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:55.453853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79848c57-9380-49f5-86ad-f7c8a0c27240 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:55.487369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac77e5-79e5-42a7-a2ce-8874d9a98271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:55.487976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac77e5-79e5-42a7-a2ce-8874d9a98271 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:55.488259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac77e5-79e5-42a7-a2ce-8874d9a98271 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:55.488533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac77e5-79e5-42a7-a2ce-8874d9a98271 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:55.489199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac77e5-79e5-42a7-a2ce-8874d9a98271 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:55.490549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac77e5-79e5-42a7-a2ce-8874d9a98271 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:55.644450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 517/1020] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:58:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:58:56.175378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1435f4-9ba7-4aee-9e9b-88c45a313943 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:56.175771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1435f4-9ba7-4aee-9e9b-88c45a313943 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:56.175946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1435f4-9ba7-4aee-9e9b-88c45a313943 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:56.177426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 504/1021] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:56.182520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:56.182808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:56.183080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:56.201609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:56.202798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:56.224613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:56.278194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 518/1022] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:56.545661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1435f4-9ba7-4aee-9e9b-88c45a313943 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:56.656850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1435f4-9ba7-4aee-9e9b-88c45a313943 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:56.691222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176dd2f-078f-4e91-9f96-dd0447ea8de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:56.691854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176dd2f-078f-4e91-9f96-dd0447ea8de3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:56.692124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176dd2f-078f-4e91-9f96-dd0447ea8de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:56.692389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176dd2f-078f-4e91-9f96-dd0447ea8de3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:56.693484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176dd2f-078f-4e91-9f96-dd0447ea8de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:56.694886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176dd2f-078f-4e91-9f96-dd0447ea8de3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:56.696633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 505/1023] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:56.702678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:56.713503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a7077-d9a4-4484-ad5a-c3858b3cc3fc None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:56.713793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a7077-d9a4-4484-ad5a-c3858b3cc3fc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:56.714074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a7077-d9a4-4484-ad5a-c3858b3cc3fc None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:56.715432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 506/1024] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:56.720295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8525-caeb-421b-a918-05374f17320f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:56.720606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8525-caeb-421b-a918-05374f17320f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:56.720865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8525-caeb-421b-a918-05374f17320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:56.739469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-475c8525-caeb-421b-a918-05374f17320f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:56.740652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-475c8525-caeb-421b-a918-05374f17320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:56.802242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:56.804818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47777b94-379f-4101-8ebc-ccc2275c5033 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:56.814408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 507/1025] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:58:56.840368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:56.840980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:56.841260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:56.841532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:56.842201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:56.843661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:56.950853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 519/1026] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:58:56 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:58:57.086671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8525-caeb-421b-a918-05374f17320f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:57.192466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8525-caeb-421b-a918-05374f17320f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:57.227936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:57.228567 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.228843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.229112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] PATH_INFO: `/v3/services/image` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.229686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:57.231030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:57.240236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:57.241197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:57.242061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 508/1027] 10.222.0.62 () {56 vars in 1105 bytes} [Tue Sep 13 09:58:57 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:57.249138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:57.261029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:57.351275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b73b6-31e8-49ec-8e87-8b32e5d5efcf None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:57.392624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:57.393254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.393538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.393809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.394445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] RBAC: Adding query filter params (name=image) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:57.394751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:57.396176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:57.401475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 520/1028] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:57.407570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:57.463406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:57.509147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c2c1-72ea-457d-99e5-033436a054bd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:57.542532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:57.543138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.543446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.543719 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.544343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] RBAC: Adding query filter params (type=image) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:57.544644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:57.546554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:57.551693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 509/1029] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:57.557358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:57.629273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:57.661105 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5e2db-0ad9-4669-8935-ff8db1c25d32 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:57.694567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f559d73-b861-4072-9253-283d4835334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:57.695186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f559d73-b861-4072-9253-283d4835334e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.695492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f559d73-b861-4072-9253-283d4835334e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.695764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f559d73-b861-4072-9253-283d4835334e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.696460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f559d73-b861-4072-9253-283d4835334e None admin] RBAC: Adding query filter params (interface=public, service_id=93c49f0a2c424018897e22ff789bc9d5, region_id=RegionOne) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:57.696769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f559d73-b861-4072-9253-283d4835334e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:57.698128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f559d73-b861-4072-9253-283d4835334e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:57.708394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 521/1030] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 09:58:57 2022] GET /identity/v3/endpoints?service_id=93c49f0a2c424018897e22ff789bc9d5&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:57.709527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:57.812070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576f2b4-3c09-4b6c-b187-0f851b04767c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:57.831229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:57.848626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992299e-e440-4dd4-b70f-f28ff9f1b72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:57.849237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992299e-e440-4dd4-b70f-f28ff9f1b72d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.849515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992299e-e440-4dd4-b70f-f28ff9f1b72d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.849785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992299e-e440-4dd4-b70f-f28ff9f1b72d None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.850382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992299e-e440-4dd4-b70f-f28ff9f1b72d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:57.851822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992299e-e440-4dd4-b70f-f28ff9f1b72d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:57.858360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 510/1031] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 09:58:57 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 09:58:57.919393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bcaf8-a567-47c2-8d9a-1f2521ce6614 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.919585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bcaf8-a567-47c2-8d9a-1f2521ce6614 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.919915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bcaf8-a567-47c2-8d9a-1f2521ce6614 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.921511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 522/1032] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:58:57.927139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:57.927467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:57.927740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:57.947393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:57.948641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:58.022768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 511/1033] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:58.294076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bcaf8-a567-47c2-8d9a-1f2521ce6614 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:58.407501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bcaf8-a567-47c2-8d9a-1f2521ce6614 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:58.442149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:58.442793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:58.443112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:58.443429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:58.446342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:58.455273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:58.460176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:58.461161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:58.462071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 523/1034] 10.222.0.62 () {56 vars in 1120 bytes} [Tue Sep 13 09:58:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:58.467730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:58.564954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7e3c5-26f6-4d8e-aa4c-8faa9e6d8fa8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:58.578676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:58.600172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22014321-e045-4367-b421-772426bed2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:58.600779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22014321-e045-4367-b421-772426bed2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:58.601190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22014321-e045-4367-b421-772426bed2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:58.601477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22014321-e045-4367-b421-772426bed2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:58.602138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22014321-e045-4367-b421-772426bed2b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:58.602440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22014321-e045-4367-b421-772426bed2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:58.603825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22014321-e045-4367-b421-772426bed2b4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:58.616929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 512/1035] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:58:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:58.618222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:58.723218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08e6f5-b6bf-45cd-9005-40f8e75a45c2 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:58.759560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:58.760210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:58.760494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:58.760766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:58.772407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:58.773856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:58.788001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:58.788928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:58.790208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:58.790208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:58.790208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:58.790988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 524/1036] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:58.795288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22014321-e045-4367-b421-772426bed2b4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:58.894316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22014321-e045-4367-b421-772426bed2b4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:58.927023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:58.927668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:58.927953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:58.928228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:58.928867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:58.929202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:58.930523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:58.942719 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 513/1037] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:58.948160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:58.997532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.057891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaf8fc-39d3-4e84-a452-4ba2fc3ca5e9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.096376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.097012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.097296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.097569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.105478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:59.106914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-21db46f6-817c-4d5e-8f59-13eb48e71734 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:58:59.118778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:59.119780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:58:59.120719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:59.123762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 525/1038] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:58:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:59.126169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.159930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.171363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bad02-b0ef-4ebc-bfba-30a75a45338b None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.171657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bad02-b0ef-4ebc-bfba-30a75a45338b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.171917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bad02-b0ef-4ebc-bfba-30a75a45338b None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.173385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 526/1039] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:58:59.178539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.178833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.179130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.198553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:58:59.200024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:58:59.236041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5786e18-cc7e-419d-806b-55a121cb29f3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.272643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 527/1040] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:58:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:58:59.272813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.273116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.273397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.273668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.274393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:59.274699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:59.276135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:59.298707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 514/1041] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:58:59.299912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.343004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.405669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b2d8-651c-445a-8e8e-7de2fc2724b1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.442476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.443343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.443773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.444192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.513094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:59.514800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:59.546234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.554631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.617355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 528/1042] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:58:59.623448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.644164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.652511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f91b-40b0-4eaa-981a-b0073b805070 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.689085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.689721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.690003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.690280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.701193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:58:59.705942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:59.706860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:58:59.708905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 515/1043] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:59.712888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.740453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2853678-fe61-454e-8962-67e76ebbc4a4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.776118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.776756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.777040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.777313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.784661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.788995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:59.790851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:58:59.808590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:58:59.809561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:58:59.811039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:58:59.811039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:58:59.811039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:58:59.811039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 529/1044] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:58:59.815887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.827821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e70a1-68c8-4259-a2d2-5eb8df20eb85 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.863300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.863908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.864187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.864461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.865110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:59.865410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:59.866761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:59.875858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 516/1045] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:59.881265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.898585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:58:59.927523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90a1c-c1ba-4f7f-a275-2cb6bb549160 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:58:59.962487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:58:59.963014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:58:59.963327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:58:59.963600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:58:59.964237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:58:59.964536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:58:59.965894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:58:59.978658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 530/1046] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:58:59.983998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:58:59.994559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036eb5fa-4c2f-4660-8750-7c2cf8165b37 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.032054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.034381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96416cf6-002b-47b0-97a4-c4b69adda727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.035335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96416cf6-002b-47b0-97a4-c4b69adda727 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.035762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96416cf6-002b-47b0-97a4-c4b69adda727 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.036260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96416cf6-002b-47b0-97a4-c4b69adda727 None admin] PATH_INFO: `/v3/users/350354a7fc884c9bb9db4142e8311eb2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.050054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96416cf6-002b-47b0-97a4-c4b69adda727 None admin] RBAC: Authorizing `identity:get_user(user_id=350354a7fc884c9bb9db4142e8311eb2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.051518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96416cf6-002b-47b0-97a4-c4b69adda727 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.053327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 517/1047] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/users/350354a7fc884c9bb9db4142e8311eb2 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:00.054412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aa339-c5fc-458d-9cc1-88b0f5928a8d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.054701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aa339-c5fc-458d-9cc1-88b0f5928a8d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.054978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aa339-c5fc-458d-9cc1-88b0f5928a8d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.056335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 518/1048] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:00.058852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aa339-c5fc-458d-9cc1-88b0f5928a8d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.084303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3262d-d52a-44fe-8114-f08d3d6cbd78 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.118049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.118671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.118946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.119243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.125673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.127114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.130482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0926af36-1bc9-405b-866a-efe7f022fd8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:59:00.138243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:00.140120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:59:00.141082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:00.144911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 531/1049] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:59:00.145908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.146193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.146462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.164373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aa339-c5fc-458d-9cc1-88b0f5928a8d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.167452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:00.168653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:00.198874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.199562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.199839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.200128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.233633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 532/1050] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:00.234456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.277606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.279070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.296316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.351860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.359974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71b8a3-d2f5-4371-8492-66cdb81eb73c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.389386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 519/1051] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:00.396463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.399490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.400111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.400395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.400672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.401377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:00.401687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.403121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.415585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 533/1052] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:00.421178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.509663 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7e69c-afd1-44a1-8522-4fe319a295a5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.526159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.529033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3578f-56cb-4967-8f90-a1a6e9d42af1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.543560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.544186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.544467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.544767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.545880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] RBAC: Authorizing `identity:get_project(project_id=4193de0e154c4d06b2fb516ff1c5ef0b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.547349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.548982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 520/1053] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:00.549736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.566860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.567561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.567844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.568111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.568831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] RBAC: Adding query filter params (role.id=a3ba0e9ce6fc4630ac55028e1746b649, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=81eb5931848749a1ad22d9d228899cda) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:00.569138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.570557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.595192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 534/1054] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/role_assignments?user.id=81eb5931848749a1ad22d9d228899cda&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=a3ba0e9ce6fc4630ac55028e1746b649 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:00.595192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.595503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.662077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbf3ca-f0d7-4a21-8024-8b34a71601f9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.692167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acae27-cd76-4023-9fe5-efe2487ad482 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.701506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.702148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.702427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.702702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] PATH_INFO: `/v3/services/image` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.703318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.704842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:00.715131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:00.716229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:00.718174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 521/1055] 10.222.0.62 () {56 vars in 1105 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:00.722941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.733653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.734368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.734714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.735054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.735882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b, user.id=350354a7fc884c9bb9db4142e8311eb2) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:00.736256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.737774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.755194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 535/1056] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/role_assignments?user.id=350354a7fc884c9bb9db4142e8311eb2&scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:00.840105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476794-89ac-44d1-82be-a1b4f6aa4f45 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:00.842847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.878231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:00.878856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:00.879141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:00.879461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:00.880104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] RBAC: Adding query filter params (name=image) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:00.880410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:00.882211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:00.893175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 522/1057] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:00.894206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:00.925250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:00.998183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5df3-add3-485d-8184-3018b0f394ec None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:01.031829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:01.032667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:01.033088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:01.033528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:01.034280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] RBAC: Adding query filter params (type=image) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:01.034720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:01.036233 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e5bde-474c-402a-b8f3-b052145682ea None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:01.046663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 536/1058] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:01.047767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:01.150463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593cb1-8715-4af9-b77c-466e2c028738 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:01.185545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ec6d5-84f2-41b0-857e-08547e828426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:01.186153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec6d5-84f2-41b0-857e-08547e828426 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:01.186436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec6d5-84f2-41b0-857e-08547e828426 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:01.186709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec6d5-84f2-41b0-857e-08547e828426 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:01.187493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ec6d5-84f2-41b0-857e-08547e828426 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:01.188756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ec6d5-84f2-41b0-857e-08547e828426 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:01.295278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 523/1059] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 09:59:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:59:02.731610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52f9b8-a53f-4e6b-a5b2-2f0d776d2d95 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:02.732695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52f9b8-a53f-4e6b-a5b2-2f0d776d2d95 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:02.732695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52f9b8-a53f-4e6b-a5b2-2f0d776d2d95 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:02.734126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 537/1060] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 09:59:02.739260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0915d-f423-42bd-8d18-04ce1a127869 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:02.739699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0915d-f423-42bd-8d18-04ce1a127869 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:02.739848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0915d-f423-42bd-8d18-04ce1a127869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:02.758023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dbb0915d-f423-42bd-8d18-04ce1a127869 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:02.759250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dbb0915d-f423-42bd-8d18-04ce1a127869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:02.780400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:02.843212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 524/1061] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:02.886392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27da17-2178-470a-88c8-51ea5b02d342 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:02.886670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27da17-2178-470a-88c8-51ea5b02d342 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:02.886937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27da17-2178-470a-88c8-51ea5b02d342 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:02.891198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 538/1062] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:02.894497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:02.895142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:02.896009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:02.914599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:02.916078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:02.981886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 525/1063] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:03.116703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27da17-2178-470a-88c8-51ea5b02d342 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.210656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.221222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27da17-2178-470a-88c8-51ea5b02d342 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.255438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.256089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.256356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.256620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.260155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.268123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:03.272971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:03.274113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:03.274973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 539/1064] 10.222.0.62 () {56 vars in 1120 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:03.280707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.349306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.372451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bc1c0-ea29-4705-bdd4-d6fe7f712fc7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.402065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7a35-bca7-4e54-8173-7f77572cc8fb None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.412481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.413112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.413391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.413659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.425183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:03.430535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:03.431500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:03.433112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 526/1065] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:03.438141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.444318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.444925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.445208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.445480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.446144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:03.446445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:03.447846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:03.457358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 540/1066] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:03.462810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.479047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.541447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509145e-80d3-4c3a-a111-0c211785aef0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.562337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459022b4-4d69-4ead-9c77-fc8f372c4203 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.575011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.575640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.576528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.576806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.577488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:03.577787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:03.579151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ad422-35b5-48f4-87e7-5405950e1e25 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:03.587263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 527/1067] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:03.588413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd35cf-f4de-44bc-85dd-7bb33854eae0 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.588691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd35cf-f4de-44bc-85dd-7bb33854eae0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.588960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd35cf-f4de-44bc-85dd-7bb33854eae0 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.590158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 528/1068] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:03.593464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd35cf-f4de-44bc-85dd-7bb33854eae0 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.596233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.600701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.601336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.601615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.601889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.613509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:03.623626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-488fbdb0-e903-4bdc-b16e-2298f96e6f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:59:03.629575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:03.630736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:59:03.632027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:59:03.632027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:03.632027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:03.632027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 541/1069] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:59:03.632323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c04670-4489-413c-b005-2650458a1a21 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.632406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c04670-4489-413c-b005-2650458a1a21 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.632668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c04670-4489-413c-b005-2650458a1a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.650013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-14c04670-4489-413c-b005-2650458a1a21 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:03.651227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-14c04670-4489-413c-b005-2650458a1a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:03.699660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd35cf-f4de-44bc-85dd-7bb33854eae0 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.709998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.717427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 542/1070] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:03.718241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c04670-4489-413c-b005-2650458a1a21 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.738305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.738943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.739242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.739518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] PATH_INFO: `/v3/users/350354a7fc884c9bb9db4142e8311eb2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.740774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] RBAC: Authorizing `identity:get_user(user_id=350354a7fc884c9bb9db4142e8311eb2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:03.742319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:03.749458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 529/1071] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/users/350354a7fc884c9bb9db4142e8311eb2 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:03.750455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.826593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c04670-4489-413c-b005-2650458a1a21 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.827681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.860780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9a51-05e4-41aa-adbd-60c6ad44d805 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:03.863199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.863834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.864117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.864390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.865027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:03.865333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:03.866696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:03.878345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 543/1072] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:03.883838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.895623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:03.896259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:03.896540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:03.896812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:03.897924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] RBAC: Authorizing `identity:get_project(project_id=4193de0e154c4d06b2fb516ff1c5ef0b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:03.899374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:03.901016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 530/1073] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:03.907006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:03.944499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:03.989923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e615e-8b86-4feb-bcbd-2fec6a163573 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.013134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765158b-0bc8-4975-bf09-dcc0c1f304d9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.022964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.023618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.023903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.024176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.032102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.033528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:59:04.045410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:04.046493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 544/1074] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:59:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:59:04.047495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.051923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.052585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.052874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.053148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b/users/350354a7fc884c9bb9db4142e8311eb2/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.056035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] RBAC: Authorizing `identity:create_grant(project_id=4193de0e154c4d06b2fb516ff1c5ef0b, user_id=350354a7fc884c9bb9db4142e8311eb2, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.057454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.106443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:04.112646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 531/1075] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:59:03 2022] PUT /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b/users/350354a7fc884c9bb9db4142e8311eb2/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:59:04.113436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.162008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf318fe-ef80-4247-9a23-dd6d0ad64ad0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.184520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:04.202682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.203352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.203638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.203913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.231552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da1cca-09c4-4452-9ea2-4a9069b52927 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.269704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.270331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.270613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.270889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.271674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:04.271987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.273450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.277111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.278566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.298655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 532/1076] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:04.305958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.370094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:04.399206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 545/1077] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:04.402056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.419712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:04.427903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf15e8c-2730-434c-851c-b1f26b4464a2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.465053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.465987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.467089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.467695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.479842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.481321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:59:04.496963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:04.498084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:59:04.503689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:59:04.503689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:04.503689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:04.503689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 533/1078] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:04.503989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.510874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74484-39a8-4794-8ece-13b4b488e4de None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.546345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.546982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.547294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.547571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.557987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.559891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:59:04.574553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:04.575565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:59:04.582051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:59:04.582051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:04.582051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:04.582051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 546/1079] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:59:04.582342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.617276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219469-42d8-43cd-9c0c-f71b3784c452 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.627309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:04.653083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:04.655934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.656543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.656996 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.657270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.658719 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:04.659033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.660439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.675218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 534/1080] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:04.682059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.697151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49561bd8-99d1-4fd9-90d0-86919c48797f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.734434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.735047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.735372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.735648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.736828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:04.737397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.738599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.751406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 547/1081] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:04.806171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff982d6-6c6a-4fb2-b570-bb8e2b4a8df2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:04.843613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:04.844264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:04.844541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:04.844808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:04.852444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:04.853898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:59:04.866556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:04.867784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:04.868763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 535/1082] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:59:04.875207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:04.986200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c8059-08dc-4a53-8ba7-23dc4fe4e626 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:05.015865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:05.023716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:05.024332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:05.024613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:05.024888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:05.025583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:05.025889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:05.027535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b987a-559d-4e71-957f-7ad3186c334d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:05.040598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 548/1083] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:05.046776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:05.081119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:05.152603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0380a5f-9744-4650-914b-00b28154b9b2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:05.185532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb55dc-c76d-4849-a96b-d45e3f233e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:05.186179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb55dc-c76d-4849-a96b-d45e3f233e90 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:05.186459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb55dc-c76d-4849-a96b-d45e3f233e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:05.186731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb55dc-c76d-4849-a96b-d45e3f233e90 None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/81eb5931848749a1ad22d9d228899cda/roles/a3ba0e9ce6fc4630ac55028e1746b649` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:05.188857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb55dc-c76d-4849-a96b-d45e3f233e90 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=81eb5931848749a1ad22d9d228899cda, role_id=a3ba0e9ce6fc4630ac55028e1746b649)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:05.190263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb55dc-c76d-4849-a96b-d45e3f233e90 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:05.259361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 536/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:59:05 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/81eb5931848749a1ad22d9d228899cda/roles/a3ba0e9ce6fc4630ac55028e1746b649 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:59:06.282923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44a3f-d103-46eb-992e-2ca93c9b31f1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:06.283915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44a3f-d103-46eb-992e-2ca93c9b31f1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:06.284186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44a3f-d103-46eb-992e-2ca93c9b31f1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:06.285538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 549/1085] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:06.291237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:06.291662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:06.291784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:06.309898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:06.311073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:06.374555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 537/1086] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:06.642676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44a3f-d103-46eb-992e-2ca93c9b31f1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:06.741779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44a3f-d103-46eb-992e-2ca93c9b31f1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:06.777789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:06.778406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:06.778681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:06.779592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:06.790222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:06.794943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:06.796067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:06.796975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 550/1087] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:06.802230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:06.819955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:06.898317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882d11-225f-4a07-97b4-a4d0f58e14ba None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:06.932044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:06.932638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:06.932918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:06.933190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:06.933849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:06.934150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:06.936131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:06.945390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 538/1088] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:06.951906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:07.001629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:07.053317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5a8f-0e21-4097-b266-57140eee48cf None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:07.088259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03025374-6dfb-4e7f-8de7-ad53b6a303c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:07.088889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025374-6dfb-4e7f-8de7-ad53b6a303c5 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.089173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025374-6dfb-4e7f-8de7-ad53b6a303c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.089470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025374-6dfb-4e7f-8de7-ad53b6a303c5 None admin] PATH_INFO: `/v3/users/350354a7fc884c9bb9db4142e8311eb2` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.090568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03025374-6dfb-4e7f-8de7-ad53b6a303c5 None admin] RBAC: Authorizing `identity:get_user(user_id=350354a7fc884c9bb9db4142e8311eb2)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:07.092046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03025374-6dfb-4e7f-8de7-ad53b6a303c5 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:07.093636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 551/1089] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:06 2022] GET /identity/v3/users/350354a7fc884c9bb9db4142e8311eb2 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:07.099468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:07.178240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:07.196724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f782b6-64eb-43e0-93ef-7f7f757415a5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:07.230048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:07.230655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.230938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.231237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.298484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:07.300009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:07.379110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:07.399199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 539/1090] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:07.405093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:07.405591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a6cf-2334-4701-99e9-c9fa9ac542cc None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.406028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a6cf-2334-4701-99e9-c9fa9ac542cc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.406128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a6cf-2334-4701-99e9-c9fa9ac542cc None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.407483 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 552/1091] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:07.412506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.412801 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.413078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.431644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:07.432781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:07.498871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 553/1092] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:59:07.503322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739f847-1c3c-4ccb-be39-476aa2fb44c2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:07.535637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:07.536260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.536543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.536818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.537929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=4193de0e154c4d06b2fb516ff1c5ef0b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:07.539385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:07.541269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 540/1093] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:07 2022] GET /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:07.547898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:07.593427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:07.645234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a573cb1-cffc-4dea-85f3-8dc05d24c196 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:07.678530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:07.679290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.679573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.679847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.680624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b, user.id=350354a7fc884c9bb9db4142e8311eb2) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:07.680937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:07.682409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:07.688814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 554/1094] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:07 2022] GET /identity/v3/role_assignments?user.id=350354a7fc884c9bb9db4142e8311eb2&scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:07.770743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:07.822225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:07.874005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3670796-0369-42f1-9f1f-58bd6cf7e6e2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:07.907618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:07.908258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:07.908537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:07.908809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:07.920072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:07.924923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:07.926011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:07.927037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 541/1095] 10.222.0.62 () {56 vars in 1120 bytes} [Tue Sep 13 09:59:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:07.931970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:08.032234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a449ab-9f64-4396-b304-def180a74be9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:08.042773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:08.066163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:08.066775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:08.067060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:08.067398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:08.068064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:08.068368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:08.069729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:08.079023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 555/1096] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:59:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:08.084753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:08.183452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac3525-c18f-4fae-8bb4-47072647b8cb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:08.192801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:08.218322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:08.219005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:08.219586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:08.220094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:08.231261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:08.232711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:59:08.246782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:08.248067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:59:08.248990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:59:08.248990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:08.248990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:08.248990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 542/1097] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:08.253933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:08.359275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf14b14-14c9-4df0-808d-fdb6571dbd0e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:08.394565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:08.395199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:08.395488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:08.395765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:08.396405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:08.396709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:08.398068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:08.411239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 556/1098] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:08.417226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:08.515185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992635be-d347-41f6-9977-439d4767d8f8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:08.547232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:08.547861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:08.548142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:08.548414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:08.555566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:08.556988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 09:59:08.568358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:08.569305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:08.570236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 543/1099] 10.222.0.62 () {56 vars in 1093 bytes} [Tue Sep 13 09:59:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:08.575194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:08.608746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:08.682227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f7a80-d82a-451f-bd2e-8bc1f182bcb7 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:08.714055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:08.714647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:08.714924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:08.715219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:08.715929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:08.716743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:08.718157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:08.735027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 557/1100] 10.222.0.62 () {56 vars in 1133 bytes} [Tue Sep 13 09:59:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:08.740725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:08.762804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:08.842069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f503f-111f-4de7-be66-49027205c220 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:08.874814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:08.875449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:08.875731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:08.876005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:08.941795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:08.943174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:08.963610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:09.042507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 544/1101] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:09.047978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:09.137486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:09.147410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ade4c-04f1-4dfe-9765-4f72ceaa6ad8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:09.181044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:09.181674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.181954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.182225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.192566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:09.193936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:59:09.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:09.209634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 09:59:09.210541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:59:09.210541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 09:59:09.210541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:09.210541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 558/1102] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:09.215577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:09.311995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85498f-cab7-48d8-902c-9f91bdd92aaa None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:09.331565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:09.345387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:09.345991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.346268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.348312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.348312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:09.348312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:09.350366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:09.364602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 545/1103] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 09:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:09.370282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:09.474277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd10ed-281e-4966-945b-261a8f2be2a4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:09.507398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:09.508027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.508309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.508585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.515820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:09.517251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 09:59:09.528317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:09.529221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:09.530720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 559/1104] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 09:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 09:59:09.535216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:09.633056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10731eb3-c577-4bad-8bfb-8e1c79a121e5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:09.666865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:09.667498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.667781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.668053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.668739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:09.669045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:09.670986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b635c3-a86c-4441-9262-53131d2e18c6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:09.687308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 546/1105] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 09:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:09.693923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:09.696936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:09.796710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018464-9581-419f-a8b7-4bd51ca4facc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:09.830096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:09.830693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.830974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.831274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.831990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] RBAC: Adding query filter params (role.id=a3ba0e9ce6fc4630ac55028e1746b649, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=81eb5931848749a1ad22d9d228899cda) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:09.832296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:09.833712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952810f1-0bff-46b5-b5e2-9ed0344a64f3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:09.839352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 560/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:09 2022] GET /identity/v3/role_assignments?user.id=81eb5931848749a1ad22d9d228899cda&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=a3ba0e9ce6fc4630ac55028e1746b649 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:09.885326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1bb57-db9e-4528-9b5c-c6429a44ed46 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.885610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1bb57-db9e-4528-9b5c-c6429a44ed46 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.885882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1bb57-db9e-4528-9b5c-c6429a44ed46 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.887266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 547/1107] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:09.893299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:09.893591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:09.893872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:09.912970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:09.914219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:09.979076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:09.987942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 561/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:10.262102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1bb57-db9e-4528-9b5c-c6429a44ed46 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:10.309049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:10.368341 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1bb57-db9e-4528-9b5c-c6429a44ed46 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:10.405780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3f76-cb94-419f-bae9-af1fa6b5b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:10.406411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3f76-cb94-419f-bae9-af1fa6b5b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:10.406686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3f76-cb94-419f-bae9-af1fa6b5b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:10.406958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3f76-cb94-419f-bae9-af1fa6b5b7e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:10.408120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3f76-cb94-419f-bae9-af1fa6b5b7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:10.409541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3f76-cb94-419f-bae9-af1fa6b5b7e0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:10.411114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 548/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:10.417458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:10.517348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a3c66-89b2-48a9-8425-41f69e4a8ca6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:10.551991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c8b18-1972-44b4-a446-d4140625f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:10.552600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c8b18-1972-44b4-a446-d4140625f58e None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:10.552882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c8b18-1972-44b4-a446-d4140625f58e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:10.553156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c8b18-1972-44b4-a446-d4140625f58e None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:10.553815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c8b18-1972-44b4-a446-d4140625f58e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:10.555272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c8b18-1972-44b4-a446-d4140625f58e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:10.664517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 562/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:59:10 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:59:12.838248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17156f9-4865-4d62-a94d-2d6026f28a6b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:12.838799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17156f9-4865-4d62-a94d-2d6026f28a6b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:12.838799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17156f9-4865-4d62-a94d-2d6026f28a6b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:12.840188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 549/1111] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:12.845455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:12.845888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:12.845997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:12.864093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:12.865266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:12.909912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:12.929350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 563/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:13.196778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17156f9-4865-4d62-a94d-2d6026f28a6b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:13.297551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17156f9-4865-4d62-a94d-2d6026f28a6b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:13.306544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:13.330736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:13.331385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:13.331669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:13.331941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] PATH_INFO: `/v3/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:13.333310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] RBAC: Authorizing `identity:get_role(role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:13.334665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:13.336024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 550/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:13 2022] GET /identity/v3/roles/98057a16b9eb4d96a1931655b2cd15be => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:13.342355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:13.440217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf568f-4317-4cb4-bc44-20660905ea96 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:13.473991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:13.474565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:13.474844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:13.475117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] PATH_INFO: `/v3/users/e588456ebaf041989fab7173145363a3` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:13.488012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] RBAC: Authorizing `identity:get_user(user_id=e588456ebaf041989fab7173145363a3)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:13.489434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:13.491028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 564/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:13 2022] GET /identity/v3/users/e588456ebaf041989fab7173145363a3 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:13.496948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:13.594238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00b289-4c36-40fb-b2ab-4e0672b217da None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:13.626690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:13.627319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:13.627604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:13.627877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:13.671689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:13.693377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:13.694800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:13.787936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 551/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:13.794392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:13.835350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:13.893048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641085-dbb2-4a43-bf01-c474a91d6deb None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:13.925558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb445c-8cb5-4581-801e-87bc32383c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:13.926188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb445c-8cb5-4581-801e-87bc32383c30 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:13.926473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb445c-8cb5-4581-801e-87bc32383c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:13.926749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb445c-8cb5-4581-801e-87bc32383c30 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:13.927890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb445c-8cb5-4581-801e-87bc32383c30 None admin] RBAC: Authorizing `identity:get_project(project_id=4193de0e154c4d06b2fb516ff1c5ef0b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:13.929315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb445c-8cb5-4581-801e-87bc32383c30 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:13.930934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 565/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:13 2022] GET /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:13.937604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:14.010649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:14.033294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a794a-1fd1-4565-a386-c6156a6ee693 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:14.066135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:14.066672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:14.066955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:14.067258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:14.067970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] RBAC: Adding query filter params (role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b, user.id=e588456ebaf041989fab7173145363a3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:14.068281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:14.069693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af341e3-2ef8-4baf-b667-c15525c90bfa None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:14.074818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 552/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:13 2022] GET /identity/v3/role_assignments?user.id=e588456ebaf041989fab7173145363a3&scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:16.176787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50f357-5af7-45fa-b5bc-eef882cf70a4 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:16.177313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50f357-5af7-45fa-b5bc-eef882cf70a4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:16.177313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50f357-5af7-45fa-b5bc-eef882cf70a4 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:16.178678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 566/1118] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:16.183724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:16.184118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:16.184269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:16.202653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:16.204222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:16.270288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 553/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:16.534210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50f357-5af7-45fa-b5bc-eef882cf70a4 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:16.632623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50f357-5af7-45fa-b5bc-eef882cf70a4 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:16.641910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:16.666001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:16.666615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:16.666913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:16.667205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] PATH_INFO: `/v3/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:16.668565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] RBAC: Authorizing `identity:get_role(role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:16.669970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:16.671622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 567/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:16 2022] GET /identity/v3/roles/98057a16b9eb4d96a1931655b2cd15be => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:16.677137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:16.742534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:16.772888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed4655-e2b6-4f92-b1ad-b8c78e338343 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:16.805100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:16.805663 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:16.805947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:16.806231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] PATH_INFO: `/v3/users/e588456ebaf041989fab7173145363a3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:16.807372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] RBAC: Authorizing `identity:get_user(user_id=e588456ebaf041989fab7173145363a3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:16.808780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:16.810841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 554/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:16 2022] GET /identity/v3/users/e588456ebaf041989fab7173145363a3 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:16.816422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:16.914374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b863b9a-a48f-4553-82cc-32d8bb97db83 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:16.953762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f47f5-feea-4a09-9f74-2712d4df0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:16.954389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f47f5-feea-4a09-9f74-2712d4df0af3 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:16.954677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f47f5-feea-4a09-9f74-2712d4df0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:16.954953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f47f5-feea-4a09-9f74-2712d4df0af3 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:16.956106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f47f5-feea-4a09-9f74-2712d4df0af3 None admin] RBAC: Authorizing `identity:get_project(project_id=4193de0e154c4d06b2fb516ff1c5ef0b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:16.957523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f47f5-feea-4a09-9f74-2712d4df0af3 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:16.959138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 568/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:16 2022] GET /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:16.965078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:17.061746 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531e786-89c2-4340-9a9d-3ad19e877482 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:17.094384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd034d16-e314-4395-9f54-53b95c6b87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:17.095025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd034d16-e314-4395-9f54-53b95c6b87be None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:17.095333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd034d16-e314-4395-9f54-53b95c6b87be None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:17.095605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd034d16-e314-4395-9f54-53b95c6b87be None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b/users/e588456ebaf041989fab7173145363a3/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:17.097711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd034d16-e314-4395-9f54-53b95c6b87be None admin] RBAC: Authorizing `identity:create_grant(project_id=4193de0e154c4d06b2fb516ff1c5ef0b, user_id=e588456ebaf041989fab7173145363a3, role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:17.099117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd034d16-e314-4395-9f54-53b95c6b87be None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:17.151236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:17.156925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 555/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:59:16 2022] PUT /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b/users/e588456ebaf041989fab7173145363a3/roles/98057a16b9eb4d96a1931655b2cd15be => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:59:19.269641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139a099-59f7-4ff3-b63e-a121b927591a None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:19.270193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139a099-59f7-4ff3-b63e-a121b927591a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:19.270193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139a099-59f7-4ff3-b63e-a121b927591a None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:19.271535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 569/1124] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:19.276691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:19.277062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:19.277247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:19.296399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:19.297601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:19.362351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 556/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:59:19.629331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139a099-59f7-4ff3-b63e-a121b927591a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:19.675029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:19.734918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139a099-59f7-4ff3-b63e-a121b927591a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:19.768662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:19.769275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:19.769549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:19.769816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] PATH_INFO: `/v3/roles/98057a16b9eb4d96a1931655b2cd15be` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:19.771204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] RBAC: Authorizing `identity:get_role(role_id=98057a16b9eb4d96a1931655b2cd15be)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:19.772551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:19.773851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 570/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:19 2022] GET /identity/v3/roles/98057a16b9eb4d96a1931655b2cd15be => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:19.779697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:19.873449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6091a-9b26-480f-b374-97f0a29a5105 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:19.898753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:19.906172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:19.906787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:19.907065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:19.907710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] PATH_INFO: `/v3/users/e588456ebaf041989fab7173145363a3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:19.908827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=e588456ebaf041989fab7173145363a3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:19.910232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:19.911805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 557/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:19 2022] GET /identity/v3/users/e588456ebaf041989fab7173145363a3 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:19.917725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:20.015416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd60d1-ddbf-4afa-a82d-f1e88cd1a7c6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:20.028270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:20.048881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:20.049478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:20.049763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:20.050036 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:20.132259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:20.133700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:20.226862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 571/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:20.233426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:20.330347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b4d7-dd81-455a-bdbd-7fd1ba5ce5c2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:20.363214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d57383-a227-497b-ba28-e8251caea8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:20.367461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d57383-a227-497b-ba28-e8251caea8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:20.367755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d57383-a227-497b-ba28-e8251caea8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:20.368037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d57383-a227-497b-ba28-e8251caea8b5 None admin] PATH_INFO: `/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:20.369191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d57383-a227-497b-ba28-e8251caea8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4193de0e154c4d06b2fb516ff1c5ef0b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:20.370621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d57383-a227-497b-ba28-e8251caea8b5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:20.372840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 558/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:20 2022] GET /identity/v3/projects/4193de0e154c4d06b2fb516ff1c5ef0b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:20.379094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:20.393139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:20.476042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd771d-5757-4f52-975d-9413605f1dca None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:20.510678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:20.511300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:20.511584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:20.511859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:20.512588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] RBAC: Adding query filter params (role.id=98057a16b9eb4d96a1931655b2cd15be, scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b, user.id=e588456ebaf041989fab7173145363a3) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:20.512895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:20.514308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae91b-7c83-490c-aea0-38e79a21f238 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:20.519931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 572/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:20 2022] GET /identity/v3/role_assignments?user.id=e588456ebaf041989fab7173145363a3&scope.project.id=4193de0e154c4d06b2fb516ff1c5ef0b&role.id=98057a16b9eb4d96a1931655b2cd15be => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:22.663690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5aeb0-5070-4048-8320-dbc1f7b80c76 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:22.664184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5aeb0-5070-4048-8320-dbc1f7b80c76 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:22.664184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5aeb0-5070-4048-8320-dbc1f7b80c76 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:22.665755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 559/1131] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:22.670987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:22.671435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:22.671562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:22.689716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:22.690902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:22.755410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 573/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:23.020771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5aeb0-5070-4048-8320-dbc1f7b80c76 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:23.081741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:23.117976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5aeb0-5070-4048-8320-dbc1f7b80c76 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:23.151492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4a4b-ea1a-473f-a45d-1f725df0c630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:23.152112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4a4b-ea1a-473f-a45d-1f725df0c630 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:23.152394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4a4b-ea1a-473f-a45d-1f725df0c630 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:23.152667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4a4b-ea1a-473f-a45d-1f725df0c630 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:23.153779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d4a4b-ea1a-473f-a45d-1f725df0c630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:23.155205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d4a4b-ea1a-473f-a45d-1f725df0c630 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:23.156767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 560/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:23.162500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:23.171255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:23.260223 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0c842-0825-4ef4-963b-3247d2374c5d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:23.298219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7a5c3-ed45-4716-830c-9ab687864dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:23.298760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7a5c3-ed45-4716-830c-9ab687864dc9 None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:23.299043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7a5c3-ed45-4716-830c-9ab687864dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:23.299346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7a5c3-ed45-4716-830c-9ab687864dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:23.300002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7a5c3-ed45-4716-830c-9ab687864dc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:23.301452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7a5c3-ed45-4716-830c-9ab687864dc9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:23.380325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 574/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:59:23 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:59:25.539286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91418d-70aa-4a58-a06e-e08011fc47f2 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:25.540077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91418d-70aa-4a58-a06e-e08011fc47f2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:25.540077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91418d-70aa-4a58-a06e-e08011fc47f2 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:25.541422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 561/1135] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:25.547311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:25.547691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:25.547868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:25.566320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:25.567548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:25.618351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:25.632198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 575/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:25.901716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91418d-70aa-4a58-a06e-e08011fc47f2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:26.000475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91418d-70aa-4a58-a06e-e08011fc47f2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:26.028970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:26.033937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446419e8-06e2-4065-b7f0-e762086739fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:26.035399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446419e8-06e2-4065-b7f0-e762086739fb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:26.035399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446419e8-06e2-4065-b7f0-e762086739fb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:26.035399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446419e8-06e2-4065-b7f0-e762086739fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:26.036322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446419e8-06e2-4065-b7f0-e762086739fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:26.037744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446419e8-06e2-4065-b7f0-e762086739fb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:26.039322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 562/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 09:59:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:26.046066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:26.142134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67747-642f-4ee8-87f5-e39bc5771a1c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:26.175946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4212bd-aa17-4055-b548-d26cab26095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:26.176487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4212bd-aa17-4055-b548-d26cab26095a None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:26.176767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4212bd-aa17-4055-b548-d26cab26095a None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:26.177038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4212bd-aa17-4055-b548-d26cab26095a None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:26.177691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4212bd-aa17-4055-b548-d26cab26095a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:26.179216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4212bd-aa17-4055-b548-d26cab26095a None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:26.246897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 576/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:59:26 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:59:28.422237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eae1f-d3f5-488c-bde6-1f0dd8a48c70 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:28.422706 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eae1f-d3f5-488c-bde6-1f0dd8a48c70 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:28.422812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eae1f-d3f5-488c-bde6-1f0dd8a48c70 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:28.424153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 563/1139] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:28.429334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:28.429757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:28.429890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:28.448470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:28.449669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:28.488699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:28.514859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 577/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:28.782779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eae1f-d3f5-488c-bde6-1f0dd8a48c70 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:28.880645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eae1f-d3f5-488c-bde6-1f0dd8a48c70 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:28.913201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:28.913810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:28.914089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:28.914358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:28.924920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:28.929684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:28.930923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:28.932111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 564/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:28.936672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:29.032398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97723-83c3-4a0b-b4f3-ed8c9613ddb7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:29.065728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:29.066267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:29.066549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:29.066823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:29.067509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:29.067815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:29.069174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:29.077426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 578/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:29.083220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:29.142070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:29.181153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7812a-7a79-4e3e-9ef4-796e7290253d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:29.213519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:29.217106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:29.217106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:29.217106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] PATH_INFO: `/v3/users/5a3a68911b584ffd991742a461b24a6f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:29.227808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] RBAC: Authorizing `identity:get_user(user_id=5a3a68911b584ffd991742a461b24a6f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:29.229276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:29.230852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 565/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:29 2022] GET /identity/v3/users/5a3a68911b584ffd991742a461b24a6f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:29.236750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:29.271394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:29.333756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576877-6f78-4463-a82c-51169a7352d4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:29.366064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:29.366653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:29.366934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:29.367229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:29.433449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:29.435419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:29.467569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:29.530288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 579/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:29.536237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:29.632051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341146-a933-4fb7-be67-1f3560e880d4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:29.648580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:29.665195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310663-8d58-4b47-9d10-9a8ccf5a2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:29.665807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310663-8d58-4b47-9d10-9a8ccf5a2e01 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:29.666084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310663-8d58-4b47-9d10-9a8ccf5a2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:29.666356 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310663-8d58-4b47-9d10-9a8ccf5a2e01 None admin] PATH_INFO: `/v3/projects/c5f31613b6bc41718509bf1efb9811fc` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:29.667493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43310663-8d58-4b47-9d10-9a8ccf5a2e01 None admin] RBAC: Authorizing `identity:get_project(project_id=c5f31613b6bc41718509bf1efb9811fc)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:29.669548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43310663-8d58-4b47-9d10-9a8ccf5a2e01 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:29.671141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 566/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:29 2022] GET /identity/v3/projects/c5f31613b6bc41718509bf1efb9811fc => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:29.677147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:29.771878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c166c85-5145-4852-bb0b-278077ec218e None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:29.805601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:29.806186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:29.806466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:29.806739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:29.807481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=c5f31613b6bc41718509bf1efb9811fc, user.id=5a3a68911b584ffd991742a461b24a6f) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:29.807792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:29.809293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4a336-2de4-493f-997c-4a19c617f19d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:29.812069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:29.814854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 580/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:29 2022] GET /identity/v3/role_assignments?user.id=5a3a68911b584ffd991742a461b24a6f&scope.project.id=c5f31613b6bc41718509bf1efb9811fc&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:31.927483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839fa90-53db-4145-8be6-6995f80393cb None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:31.928011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839fa90-53db-4145-8be6-6995f80393cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:31.928011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839fa90-53db-4145-8be6-6995f80393cb None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:31.929385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 567/1147] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:31.934943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:31.935357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:31.935520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:31.953819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:31.955010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:32.019218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 581/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:32.292668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839fa90-53db-4145-8be6-6995f80393cb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:32.392276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839fa90-53db-4145-8be6-6995f80393cb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:32.425417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:32.426037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:32.426319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:32.426588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:32.436936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:32.441669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:32.442761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:32.443972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 568/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:32.448752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:32.543237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52e597-20ec-4124-a20d-f1070d28c26a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:32.570919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:32.576654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:32.577258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:32.577539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:32.577813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:32.578473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:32.578805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:32.580211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:32.587935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 582/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:32.594256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:32.642245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:32.693926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c707-da32-4b96-b9bc-7aa2bb573ac5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:32.726061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:32.726677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:32.726959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:32.727334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] PATH_INFO: `/v3/users/5a3a68911b584ffd991742a461b24a6f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:32.728451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] RBAC: Authorizing `identity:get_user(user_id=5a3a68911b584ffd991742a461b24a6f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:32.729865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:32.732067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 569/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:32 2022] GET /identity/v3/users/5a3a68911b584ffd991742a461b24a6f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:32.737857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:32.833069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e23908-d05b-4add-a5cc-b00bf3f7d9d0 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:32.864798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63948318-f476-4fc8-9a29-a717c21650a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:32.865412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63948318-f476-4fc8-9a29-a717c21650a6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:32.865688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63948318-f476-4fc8-9a29-a717c21650a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:32.865957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63948318-f476-4fc8-9a29-a717c21650a6 None admin] PATH_INFO: `/v3/projects/c5f31613b6bc41718509bf1efb9811fc` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:32.867045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63948318-f476-4fc8-9a29-a717c21650a6 None admin] RBAC: Authorizing `identity:get_project(project_id=c5f31613b6bc41718509bf1efb9811fc)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:32.868486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63948318-f476-4fc8-9a29-a717c21650a6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:32.870064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 583/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:32 2022] GET /identity/v3/projects/c5f31613b6bc41718509bf1efb9811fc => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:32.875825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:32.974326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696971-5a47-4b51-ac55-a3f1a7562135 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:32.987451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:33.006381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6a67b-09d3-4c0c-91f3-2e4d4983f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:33.007024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a67b-09d3-4c0c-91f3-2e4d4983f186 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:33.007342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a67b-09d3-4c0c-91f3-2e4d4983f186 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:33.007619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a67b-09d3-4c0c-91f3-2e4d4983f186 None admin] PATH_INFO: `/v3/projects/c5f31613b6bc41718509bf1efb9811fc/users/5a3a68911b584ffd991742a461b24a6f/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:33.009501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6a67b-09d3-4c0c-91f3-2e4d4983f186 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5f31613b6bc41718509bf1efb9811fc, user_id=5a3a68911b584ffd991742a461b24a6f, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:33.010900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6a67b-09d3-4c0c-91f3-2e4d4983f186 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:33.080068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 570/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:59:32 2022] PUT /identity/v3/projects/c5f31613b6bc41718509bf1efb9811fc/users/5a3a68911b584ffd991742a461b24a6f/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:59:35.194063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8bef-d375-44ca-a9d4-e40f5fb333c9 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:35.194596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8bef-d375-44ca-a9d4-e40f5fb333c9 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:35.194596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8bef-d375-44ca-a9d4-e40f5fb333c9 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:35.196995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 584/1154] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:35.202326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:35.202722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:35.202879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:35.220521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:35.221716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:35.286405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 571/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:35.556794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8bef-d375-44ca-a9d4-e40f5fb333c9 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:35.615182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:35.655395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8bef-d375-44ca-a9d4-e40f5fb333c9 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:35.688067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:35.688675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:35.688951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:35.689219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:35.698940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:35.704124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:35.705204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:35.706239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 585/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:35.711210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:35.808428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157ee8-48a9-452b-a7a7-2c3065d6aac2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:35.810540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:35.846465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:35.847068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:35.847376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:35.847651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:35.848308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:35.848631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:35.850003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:35.858208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 572/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:35.864253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:35.961265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bbc7a-a8ae-406c-b8b3-8b89f1d7e98d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:35.980784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:35.994230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:35.994847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:35.995129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:35.995434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] PATH_INFO: `/v3/users/5a3a68911b584ffd991742a461b24a6f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:35.996581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] RBAC: Authorizing `identity:get_user(user_id=5a3a68911b584ffd991742a461b24a6f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:35.998018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:35.999677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 586/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:35 2022] GET /identity/v3/users/5a3a68911b584ffd991742a461b24a6f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:36.005314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:36.103564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275f98-8abf-4607-aaec-979cc68202e5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:36.137783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:36.138378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:36.138658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:36.138930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:36.171426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:36.205096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:36.206528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:36.301504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 573/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:36.307635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:36.405254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a71fc-636f-4086-824c-d8339e9024c1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:36.438314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d1c3a-8086-46ee-ab5d-59d9c4836c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:36.438940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1c3a-8086-46ee-ab5d-59d9c4836c2b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:36.439246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1c3a-8086-46ee-ab5d-59d9c4836c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:36.439522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1c3a-8086-46ee-ab5d-59d9c4836c2b None admin] PATH_INFO: `/v3/projects/c5f31613b6bc41718509bf1efb9811fc` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:36.440629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d1c3a-8086-46ee-ab5d-59d9c4836c2b None admin] RBAC: Authorizing `identity:get_project(project_id=c5f31613b6bc41718509bf1efb9811fc)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:36.442046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d1c3a-8086-46ee-ab5d-59d9c4836c2b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:36.443719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 587/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:36 2022] GET /identity/v3/projects/c5f31613b6bc41718509bf1efb9811fc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:36.449815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:36.513067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:36.551147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706ef49-7a12-44fb-9ba3-6b9d2e3e89ec None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:36.584554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:36.585145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:36.585426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:36.585699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:36.586410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=c5f31613b6bc41718509bf1efb9811fc, user.id=5a3a68911b584ffd991742a461b24a6f) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:36.586721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:36.588155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb494da-61fc-4354-b1f5-c645f792b571 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:36.593743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 574/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:36 2022] GET /identity/v3/role_assignments?user.id=5a3a68911b584ffd991742a461b24a6f&scope.project.id=c5f31613b6bc41718509bf1efb9811fc&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:38.765883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685944b-9f68-48c6-9a28-5b6cf3b38841 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:38.766427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685944b-9f68-48c6-9a28-5b6cf3b38841 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:38.766427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685944b-9f68-48c6-9a28-5b6cf3b38841 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:38.767798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 588/1162] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:38.773112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:38.773506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:38.773670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:38.792311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:38.793502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:38.858477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 575/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:39.131327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685944b-9f68-48c6-9a28-5b6cf3b38841 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:39.188588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:39.232480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685944b-9f68-48c6-9a28-5b6cf3b38841 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:39.264506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:39.265101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:39.265382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:39.265652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:39.329959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:39.331420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:39.384447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:39.428253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 589/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:39.434204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:39.491080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:39.530643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824f09-cf6f-4e5b-91f6-238409c94b4f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:39.571043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:39.571703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:39.571989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:39.572264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:39.579296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:39.580728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5f899d03-69e8-4225-ae65-f810f5c3ae6d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:39.592471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:39.593690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:39.594623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 576/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 09:59:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:39.599781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:39.694424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ac547-1501-48d9-b6df-9b734738c3f9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:39.728728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:39.729321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:39.729601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:39.729873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:39.730558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:39.730862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:39.732247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3599da7-f818-47f8-bbd3-fc32d047cb58 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:39.739247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 590/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Tue Sep 13 09:59:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:41.881952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ce755-cdbb-4e1c-acbb-24fdabd36cf2 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:41.882473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ce755-cdbb-4e1c-acbb-24fdabd36cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:41.882473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ce755-cdbb-4e1c-acbb-24fdabd36cf2 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:41.883874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 577/1167] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:41.889198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb990e7a-2a70-4393-9f26-489f60769ddd None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:41.889616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb990e7a-2a70-4393-9f26-489f60769ddd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:41.889750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb990e7a-2a70-4393-9f26-489f60769ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:41.907470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-cb990e7a-2a70-4393-9f26-489f60769ddd None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:41.908661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cb990e7a-2a70-4393-9f26-489f60769ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:41.922880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:41.973555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 591/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:42.249206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ce755-cdbb-4e1c-acbb-24fdabd36cf2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:42.349177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ce755-cdbb-4e1c-acbb-24fdabd36cf2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:42.370628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:42.386205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0a6e-c629-4dff-b030-ec662246cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:42.386773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0a6e-c629-4dff-b030-ec662246cb9d None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:42.387048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0a6e-c629-4dff-b030-ec662246cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:42.387350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0a6e-c629-4dff-b030-ec662246cb9d None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:42.388022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc0a6e-c629-4dff-b030-ec662246cb9d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:42.389419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc0a6e-c629-4dff-b030-ec662246cb9d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:42.442055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 578/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Tue Sep 13 09:59:42 2022] POST /identity/v3/domains => generated 267 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 09:59:44.579457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52245f-67df-42a3-9ec1-be67b76028a5 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:44.580008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52245f-67df-42a3-9ec1-be67b76028a5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:44.580008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52245f-67df-42a3-9ec1-be67b76028a5 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:44.581359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 592/1170] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:44.586590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:44.586974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:44.587137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:44.604722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:44.605928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:44.670289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 579/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:44.940654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52245f-67df-42a3-9ec1-be67b76028a5 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:45.037871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52245f-67df-42a3-9ec1-be67b76028a5 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:45.071502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fc51d-7905-4b19-9d68-77ae91d9aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:45.072120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fc51d-7905-4b19-9d68-77ae91d9aabd None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:45.072402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fc51d-7905-4b19-9d68-77ae91d9aabd None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:45.072676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fc51d-7905-4b19-9d68-77ae91d9aabd None admin] PATH_INFO: `/v3/domains/d2db632d84ba43b69b8fed04105d1bc4` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:45.082848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fc51d-7905-4b19-9d68-77ae91d9aabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=d2db632d84ba43b69b8fed04105d1bc4)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:45.084205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fc51d-7905-4b19-9d68-77ae91d9aabd None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:45.085798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 593/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Tue Sep 13 09:59:44 2022] GET /identity/v3/domains/d2db632d84ba43b69b8fed04105d1bc4 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:45.091669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:45.186761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e811676-eaad-4393-81e2-0ab56b2eedab None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:45.196839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:45.219473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37ee2c-db17-486a-99d8-4a0111ff114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:45.220065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37ee2c-db17-486a-99d8-4a0111ff114c None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:45.220341 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37ee2c-db17-486a-99d8-4a0111ff114c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:45.220609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37ee2c-db17-486a-99d8-4a0111ff114c None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:45.221245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37ee2c-db17-486a-99d8-4a0111ff114c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:45.222649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37ee2c-db17-486a-99d8-4a0111ff114c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:45.283253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 580/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 09:59:45 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:59:47.470769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f07cb-60f6-4e6b-b466-9f3733fe40e1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:47.471277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f07cb-60f6-4e6b-b466-9f3733fe40e1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:47.471380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f07cb-60f6-4e6b-b466-9f3733fe40e1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:47.472683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 594/1174] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:47.477814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:47.478243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:47.478376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:47.497357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:47.498553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:47.564215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 581/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:47.836222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f07cb-60f6-4e6b-b466-9f3733fe40e1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:47.849344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:47.934940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f07cb-60f6-4e6b-b466-9f3733fe40e1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:47.967814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536379f-6fd1-4414-8899-33a597b60283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:47.968433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536379f-6fd1-4414-8899-33a597b60283 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:47.968711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536379f-6fd1-4414-8899-33a597b60283 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:47.968979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536379f-6fd1-4414-8899-33a597b60283 None admin] PATH_INFO: `/v3/domains/d2db632d84ba43b69b8fed04105d1bc4` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:47.970066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536379f-6fd1-4414-8899-33a597b60283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d2db632d84ba43b69b8fed04105d1bc4)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:47.971507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536379f-6fd1-4414-8899-33a597b60283 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:47.973069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 595/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Tue Sep 13 09:59:47 2022] GET /identity/v3/domains/d2db632d84ba43b69b8fed04105d1bc4 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:47.978740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:48.065102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:48.073376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793da19-285e-4dae-8a04-505a10f4576f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:48.107109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5f33d-333c-4449-a745-7b45df7e9d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:48.107709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5f33d-333c-4449-a745-7b45df7e9d60 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:48.108040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5f33d-333c-4449-a745-7b45df7e9d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:48.108358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5f33d-333c-4449-a745-7b45df7e9d60 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:48.108999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5f33d-333c-4449-a745-7b45df7e9d60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:48.110406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5f33d-333c-4449-a745-7b45df7e9d60 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:48.201662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 582/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 09:59:47 2022] POST /identity/v3/users => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 09:59:50.361390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed46d7-fe40-486d-9d85-267fe20ea033 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:50.361944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed46d7-fe40-486d-9d85-267fe20ea033 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:50.361944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed46d7-fe40-486d-9d85-267fe20ea033 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:50.363320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 596/1178] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:50.368530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:50.368920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:50.369081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:50.388255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:50.389826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:50.457635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 583/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:50.731380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed46d7-fe40-486d-9d85-267fe20ea033 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:50.810590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:50.832471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed46d7-fe40-486d-9d85-267fe20ea033 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:50.865371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:50.865992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:50.866276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:50.866547 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:50.876865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:50.881628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:50.882731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:50.883947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 597/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:50.889214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:50.966470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:50.986411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0ce8a-b5f5-4d0d-9dd7-acb18ee8dc65 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:51.018910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:51.019595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:51.019914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:51.020184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:51.020832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:51.021131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:51.022470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:51.031291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 584/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:51.037209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:51.135464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98722cc-aefb-43df-b6a9-6ad803012c48 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:51.168762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:51.169380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:51.169660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:51.169932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] PATH_INFO: `/v3/users/65dc4f6712ca4feb94469aac51360b77` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:51.178248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:51.184232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] RBAC: Authorizing `identity:get_user(user_id=65dc4f6712ca4feb94469aac51360b77)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:51.189386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:51.189386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 598/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:51 2022] GET /identity/v3/users/65dc4f6712ca4feb94469aac51360b77 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:51.192990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:51.292923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d432c-c119-4830-a835-31ee82dfb141 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:51.325227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:51.325821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:51.326101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:51.326369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:51.336218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:51.391002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:51.392474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:51.487036 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 585/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:51.493017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:51.590314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c043-51ea-41b5-8b18-85758577bb3b None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:51.623557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d48baa-fd2c-465d-8bd0-f5a2c50ccf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:51.624174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d48baa-fd2c-465d-8bd0-f5a2c50ccf0b None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:51.624456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d48baa-fd2c-465d-8bd0-f5a2c50ccf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:51.624728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d48baa-fd2c-465d-8bd0-f5a2c50ccf0b None admin] PATH_INFO: `/v3/projects/27c5dee60cf9482a89635055c35c6aa9` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:51.625837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d48baa-fd2c-465d-8bd0-f5a2c50ccf0b None admin] RBAC: Authorizing `identity:get_project(project_id=27c5dee60cf9482a89635055c35c6aa9)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:51.627289 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d48baa-fd2c-465d-8bd0-f5a2c50ccf0b None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:51.628896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 599/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:51 2022] GET /identity/v3/projects/27c5dee60cf9482a89635055c35c6aa9 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:51.634871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:51.675050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:51.730953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9dbb0-df3a-42ae-9696-96d8bbf5898d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:51.763952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:51.764543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:51.764823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:51.765095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:51.765853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=27c5dee60cf9482a89635055c35c6aa9, user.id=65dc4f6712ca4feb94469aac51360b77) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:51.766162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:51.767602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d07206-5a5a-4508-8828-a30572821758 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:51.772753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 586/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:51 2022] GET /identity/v3/role_assignments?user.id=65dc4f6712ca4feb94469aac51360b77&scope.project.id=27c5dee60cf9482a89635055c35c6aa9&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 09:59:53.853626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82974-c1c5-486e-b5db-863bf749ff14 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:53.854149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82974-c1c5-486e-b5db-863bf749ff14 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:53.854149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82974-c1c5-486e-b5db-863bf749ff14 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:53.855537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 600/1186] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:53.860574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:53.860946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:53.861115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:53.878647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:53.879856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:53.943188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 587/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 09:59:54.207250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82974-c1c5-486e-b5db-863bf749ff14 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:54.269053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:54.305733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82974-c1c5-486e-b5db-863bf749ff14 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:54.338659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:54.339321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:54.339600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:54.339871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:54.349630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:54.354309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:54.355448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 09:59:54.356512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 601/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:54.361359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:54.396337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:54.464226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c594c-9def-48fd-a0a1-c0ffc5cccac5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:54.497931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:54.498539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:54.498822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:54.499096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:54.499786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:54.500092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:54.501447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:54.509200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 588/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:54.515706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:54.614968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819abb9-e543-49dd-940f-4e60f048fcc1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:54.637582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:54.647716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:54.648335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:54.648616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:54.648886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] PATH_INFO: `/v3/users/65dc4f6712ca4feb94469aac51360b77` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:54.649978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] RBAC: Authorizing `identity:get_user(user_id=65dc4f6712ca4feb94469aac51360b77)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:54.651450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:54.653034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 602/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:54 2022] GET /identity/v3/users/65dc4f6712ca4feb94469aac51360b77 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:54.658862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:54.749427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:54.754958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adab34-7b38-44c4-9b0d-16cafe208631 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:54.787413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487108ac-1908-4166-82c5-76a78c72f07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:54.788032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487108ac-1908-4166-82c5-76a78c72f07d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:54.788314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487108ac-1908-4166-82c5-76a78c72f07d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:54.788585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487108ac-1908-4166-82c5-76a78c72f07d None admin] PATH_INFO: `/v3/projects/27c5dee60cf9482a89635055c35c6aa9` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:54.789946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487108ac-1908-4166-82c5-76a78c72f07d None admin] RBAC: Authorizing `identity:get_project(project_id=27c5dee60cf9482a89635055c35c6aa9)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:54.791395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487108ac-1908-4166-82c5-76a78c72f07d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:54.793553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 589/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:54 2022] GET /identity/v3/projects/27c5dee60cf9482a89635055c35c6aa9 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:54.799433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:54.895421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65927e-f888-4b7f-83bc-b540ef5a6444 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:54.928710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6a871-a65f-4e7f-b0d2-53c266b0eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:54.929393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a871-a65f-4e7f-b0d2-53c266b0eee2 None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:54.929677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a871-a65f-4e7f-b0d2-53c266b0eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:54.929951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a871-a65f-4e7f-b0d2-53c266b0eee2 None admin] PATH_INFO: `/v3/projects/27c5dee60cf9482a89635055c35c6aa9/users/65dc4f6712ca4feb94469aac51360b77/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:54.932417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6a871-a65f-4e7f-b0d2-53c266b0eee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=27c5dee60cf9482a89635055c35c6aa9, user_id=65dc4f6712ca4feb94469aac51360b77, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:54.934477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6a871-a65f-4e7f-b0d2-53c266b0eee2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:55.004936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 603/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 09:59:54 2022] PUT /identity/v3/projects/27c5dee60cf9482a89635055c35c6aa9/users/65dc4f6712ca4feb94469aac51360b77/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 09:59:57.149625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c184-75f2-45a9-a2fa-257f332512d9 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:57.150148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c184-75f2-45a9-a2fa-257f332512d9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:57.150148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c184-75f2-45a9-a2fa-257f332512d9 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:57.151690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 590/1193] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 09:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 09:59:57.157743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:57.158166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:57.158333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:57.176379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 09:59:57.177575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 09:59:57.223845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:57.242763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 604/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 09:59:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 09:59:57.512414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c184-75f2-45a9-a2fa-257f332512d9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:57.614508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c184-75f2-45a9-a2fa-257f332512d9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:57.646708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:57.647347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:57.647624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:57.647894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:57.657469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 09:59:57.662188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 09:59:57.663306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 09:59:57.664174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 591/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 09:59:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 09:59:57.670111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:57.765865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ed49d-9e02-4a3a-81b1-3d7a8cfc5003 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:57.798527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:57.799114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:57.799418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:57.799690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:57.800344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:57.800646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:57.802001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:57.810251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 605/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 09:59:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:57.816072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:57.841514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:57.912302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02a05f-8c0b-4b1a-938a-fb5bee0b20bd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:57.945515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:57.946126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:57.946406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:57.946679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] PATH_INFO: `/v3/users/65dc4f6712ca4feb94469aac51360b77` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:57.947801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] RBAC: Authorizing `identity:get_user(user_id=65dc4f6712ca4feb94469aac51360b77)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:57.949244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:57.950790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 592/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Tue Sep 13 09:59:57 2022] GET /identity/v3/users/65dc4f6712ca4feb94469aac51360b77 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:57.956508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:58.001108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:58.054465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756cf32-2313-4bb3-8891-0f96d56db257 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:58.086295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:58.086885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:58.087185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:58.087472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:58.152583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:58.154010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:58.196381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:58.249688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 606/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 09:59:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 09:59:58.255591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:58.344406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 09:59:58.352753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26edfad-437e-40be-b402-eb7eac305141 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:58.387027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e73f03-c55b-4255-9ad3-01e354555d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:58.387693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e73f03-c55b-4255-9ad3-01e354555d02 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:58.387976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e73f03-c55b-4255-9ad3-01e354555d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:58.388250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e73f03-c55b-4255-9ad3-01e354555d02 None admin] PATH_INFO: `/v3/projects/27c5dee60cf9482a89635055c35c6aa9` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:58.389521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e73f03-c55b-4255-9ad3-01e354555d02 None admin] RBAC: Authorizing `identity:get_project(project_id=27c5dee60cf9482a89635055c35c6aa9)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:58.390785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e73f03-c55b-4255-9ad3-01e354555d02 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:58.392462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 593/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Tue Sep 13 09:59:58 2022] GET /identity/v3/projects/27c5dee60cf9482a89635055c35c6aa9 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 09:59:58.398570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 09:59:58.496673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6de513-eee5-4b5f-9ad2-0bd9c5e2fc53 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 09:59:58.529895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 09:59:58.530488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 09:59:58.530771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 09:59:58.531046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 09:59:58.531790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=27c5dee60cf9482a89635055c35c6aa9, user.id=65dc4f6712ca4feb94469aac51360b77) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 09:59:58.532101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 09:59:58.533526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31561-7d7b-4e72-a492-8e9bc0e0d0f9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 09:59:58.539096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 607/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 09:59:58 2022] GET /identity/v3/role_assignments?user.id=65dc4f6712ca4feb94469aac51360b77&scope.project.id=27c5dee60cf9482a89635055c35c6aa9&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:00:08.380881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=20 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:00:08.536622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:10.490140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5b26f-927c-408e-b160-7afa26162b02 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:10.490718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5b26f-927c-408e-b160-7afa26162b02 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:10.490718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5b26f-927c-408e-b160-7afa26162b02 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:10.492057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 594/1201] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:10.497808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:10.498240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:10.498367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:10.516349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:10.517554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:10.582882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 608/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:10.851090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5b26f-927c-408e-b160-7afa26162b02 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:10.951356 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5b26f-927c-408e-b160-7afa26162b02 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:10.983423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:10.984041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:10.984323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:10.984594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:10.994300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:10.995763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:11.010470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:11.011703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:11.013638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:11.013638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:11.013638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:11.013638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 595/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:11.018360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:11.116389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30714992-605e-4201-aa8a-ee8b16f1c8d6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:11.145198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:11.150847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:11.151485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:11.151766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:11.152040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:11.152679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:11.152985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:11.154374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6e541-2568-4e59-9b00-b9b845e67aa6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:11.165682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 609/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:11.171997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:11.222797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:11.268845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a7f5-9fe6-4e46-a40f-c144340c4a6b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:11.300801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300213f7-5c50-4aa4-8b29-60a7ea6a05ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:11.301397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300213f7-5c50-4aa4-8b29-60a7ea6a05ba None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:11.301680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300213f7-5c50-4aa4-8b29-60a7ea6a05ba None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:11.301952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300213f7-5c50-4aa4-8b29-60a7ea6a05ba None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:11.302584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300213f7-5c50-4aa4-8b29-60a7ea6a05ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:11.304022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300213f7-5c50-4aa4-8b29-60a7ea6a05ba None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:11.434091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 596/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Tue Sep 13 10:01:11 2022] POST /identity/v3/users => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:01:13.634286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f07453-257c-4ddd-8f6a-b216894635c1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:13.634822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f07453-257c-4ddd-8f6a-b216894635c1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:13.634822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f07453-257c-4ddd-8f6a-b216894635c1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:13.636229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 610/1206] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:13.642130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:13.642556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:13.642731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:13.660476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:13.661664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:13.730022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 597/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:13.994629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f07453-257c-4ddd-8f6a-b216894635c1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:14.096306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f07453-257c-4ddd-8f6a-b216894635c1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:14.129495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:14.130115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:14.130396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:14.130666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:14.140968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 10:01:14.145695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:14.146839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:14.148173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 611/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 10:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 10:01:14.152780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:14.192493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:14.249244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01326edf-6d6c-430e-886d-76e78197ca4b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:14.281254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:14.283672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:14.283672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:14.283672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:14.283672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:14.283672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:14.284707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:14.292533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 598/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 10:01:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:14.299105 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:14.348966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:14.398443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b34272-bb2f-48fb-83ee-1b7bccafbfaf None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:14.431732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:14.432356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:14.432645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:14.432922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:14.443637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:14.445074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:14.459294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:14.466236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:14.467394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:14.467394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:14.467394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:14.467394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 612/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:14.467634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:14.562088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:14.564850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd758558-827c-4164-aaf9-e84283e3910f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:14.597981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:14.598547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:14.598824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:14.599088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:14.599748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:14.600046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:14.601389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:14.612971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 599/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:01:14.618787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:14.715649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d54c-781d-489c-8774-8cc9ebef49fa None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:14.717740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:14.752967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:14.753999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:14.754286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:14.754562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:14.762214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:14.764252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 10:01:14.775092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:14.776119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:14.777536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 613/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:14.782870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:14.879595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc71360-7b04-4dcb-985e-9bf6f41114ec None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:14.912602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:14.913203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:14.913486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:14.913759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:14.914730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:14.915059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:14.916509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:14.934483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 600/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Tue Sep 13 10:01:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:14.940588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:15.038855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edc805-e38b-40ef-a485-67b1af5f07cc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:15.071086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:15.071737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:15.072015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:15.072285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:15.102826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:15.141623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:15.143005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:15.239013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 614/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 10:01:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 10:01:15.244937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:15.264446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:15.343107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23aa4-67b1-4c9d-83bb-0c3dc05eef76 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:15.376185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:15.376801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:15.377097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:15.377409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:15.387424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:15.389442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:15.409460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:15.410560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:15.411792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:15.411792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:15.411792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:15.411792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 601/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:15.412065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:15.473281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:15.510674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8354c3-401d-4454-92ee-6a3661d04c2d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:15.545612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:15.546236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:15.546501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:15.546773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:15.547438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:15.547748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:15.549114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:15.560723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 615/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:15.566384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:15.636490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:15.667242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c13ba2-8aa5-44fd-9e6a-5e957db1592b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:15.699329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:15.699954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:15.700768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:15.701041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:15.707623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:15.709045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 10:01:15.721652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:15.722754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:15.723989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 602/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 10:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:15.728758 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:15.826869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a54988-149c-42e3-b918-30b6813052dc None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:15.848703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:15.859869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:15.860469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:15.860750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:15.861022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:15.861706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:15.862016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:15.863459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44bda4-692e-4ec0-9917-38364316efaa None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:15.875638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 616/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 10:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:15.881711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:15.979214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256a918-fb69-487e-ba62-a47a18c5f123 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:16.006343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:16.011907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:16.012499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:16.013387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:16.013679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:16.014386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=751cbd1e77824b83aa3c9c03c637fadf) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:16.014687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:16.016196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f450fad-44b3-4d29-bf9c-b77cd9f7c000 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:16.021280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 603/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 10:01:15 2022] GET /identity/v3/role_assignments?user.id=751cbd1e77824b83aa3c9c03c637fadf&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:01:18.163379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a9a0-4cc6-48c2-bb19-ca967e0f1fc2 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:18.164082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a9a0-4cc6-48c2-bb19-ca967e0f1fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:18.164082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a9a0-4cc6-48c2-bb19-ca967e0f1fc2 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:18.165627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 617/1220] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:18.171939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:18.172395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:18.172495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:18.191241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:18.192550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:18.262460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 604/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:18.547275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a9a0-4cc6-48c2-bb19-ca967e0f1fc2 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:18.652189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a9a0-4cc6-48c2-bb19-ca967e0f1fc2 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:18.687223 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:18.687859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:18.688142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:18.688417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:18.698430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 10:01:18.703477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:18.704507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:18.705716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 618/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 10:01:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:18.710652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:18.811715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c841e2-114b-4600-8732-64dc10ff3875 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:18.834474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:18.845298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:18.845905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:18.846180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:18.846451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:18.847096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:18.847423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:18.848763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:18.856462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 605/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 10:01:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:18.862368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:18.907703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:18.969457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cba8a-5fa0-4c94-b9a4-6e525840f572 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.003368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.004035 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.004319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.004595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.015033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.016525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:19.031906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:19.033001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:19.034280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:19.034280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:19.034280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:19.034280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 619/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:19.038838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:19.137258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad547ea-6298-406f-99f3-730a6c515f0e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.171412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.172012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.172540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.172540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.173203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:19.173506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.174863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.186953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 606/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:19.192961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:19.285766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:19.292393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92a581-17ba-460a-b9dc-cb202b0885a9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.326703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.327366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.327650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.327925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.335389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.336852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 10:01:19.348048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:19.349189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:19.350467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 620/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:19.355200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:19.404569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:19.455974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80b53e-7a5e-4d21-a6d7-305debbb5ddd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.487828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85627629-f322-40af-8a55-445f1a709499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.492707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85627629-f322-40af-8a55-445f1a709499 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.492707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85627629-f322-40af-8a55-445f1a709499 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.492707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85627629-f322-40af-8a55-445f1a709499 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.492707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85627629-f322-40af-8a55-445f1a709499 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:19.492707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85627629-f322-40af-8a55-445f1a709499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.492707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85627629-f322-40af-8a55-445f1a709499 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.507879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 607/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:19.513711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:19.612245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef8f4-0166-4d95-8795-c4dabe485a21 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.646288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.646846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.647128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.647432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.658302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.659823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:19.674791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:19.675937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:19.676860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:19.676860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:19.676860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:19.676860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 621/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:19.681638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85627629-f322-40af-8a55-445f1a709499 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:19.777777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85627629-f322-40af-8a55-445f1a709499 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.780623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:19.811343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.811946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.812229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.812503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.813140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:19.813443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.814793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.826301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 608/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:19.832077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:19.841956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:19.931953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7957d8a-9f7d-474f-9249-86287fc64ca1 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:19.969176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:19.969778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:19.970059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:19.970330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:19.977060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:19.978495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 10:01:19.989555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:19.990619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:19.991946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 622/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:19.996647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:20.095189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409308e-cf9f-4524-abfe-1d772d375093 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:20.129460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:20.130052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:20.130330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:20.130606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:20.131298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:20.131602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:20.132997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025cab0-a374-4a14-b9eb-2cb1e7711731 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:20.144758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 609/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 10:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:20.150873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:20.216632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:20.257740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d02cc-5a0d-436c-82d7-7c8f3d39b9f6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:20.291472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8d927-4321-4197-b76c-e4c0e93808fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:20.292128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8d927-4321-4197-b76c-e4c0e93808fb None admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:20.292409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8d927-4321-4197-b76c-e4c0e93808fb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:20.292684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8d927-4321-4197-b76c-e4c0e93808fb None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/751cbd1e77824b83aa3c9c03c637fadf/roles/9a6b9c222c284a98b8c786285c749f03` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:20.307048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8d927-4321-4197-b76c-e4c0e93808fb None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=751cbd1e77824b83aa3c9c03c637fadf, role_id=9a6b9c222c284a98b8c786285c749f03)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:20.308520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8d927-4321-4197-b76c-e4c0e93808fb None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:20.358568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 623/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 10:01:20 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/751cbd1e77824b83aa3c9c03c637fadf/roles/9a6b9c222c284a98b8c786285c749f03 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:01:22.513792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fceb57-d5be-4f88-86ee-959ba2211555 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:22.514332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fceb57-d5be-4f88-86ee-959ba2211555 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:22.514332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fceb57-d5be-4f88-86ee-959ba2211555 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:22.515797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 610/1233] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:22.520999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:22.521378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:22.521551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:22.539619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:22.540804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:22.606581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 624/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:22.875550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fceb57-d5be-4f88-86ee-959ba2211555 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:22.900484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:22.973925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fceb57-d5be-4f88-86ee-959ba2211555 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.006594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.007244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.007525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.007821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:23.018564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 10:01:23.023368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:23.024452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:23.025334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 611/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Tue Sep 13 10:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:23.030658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:23.078303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:23.132365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba5f82-4351-4c9e-ac12-ee932dcf63b7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.165490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.166093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.166375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.166728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:23.167415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] RBAC: Adding query filter params (name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:23.167729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:23.169604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:23.177172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 625/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Tue Sep 13 10:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:23.183211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:23.265467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:23.280923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0626480-e000-4a21-a94a-c37c70120ffa None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.316370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.316994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.317312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.317589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:23.327470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:23.328928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:23.343191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:23.349051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:23.350159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:23.350159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:23.350159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:23.350159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 612/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:23.350445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:23.452734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7ee01-8478-4b85-b06a-e8965c6e0689 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.461367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:23.486038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.486643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.486923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.487221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:23.487851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:23.488176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:23.489529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:23.501215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 626/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:23.507083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:23.604269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb6e9a-3560-4ae2-a4a6-a356d491d03b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.637030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.638198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.638474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.638747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:23.641364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:23.646139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:23.647601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 10:01:23.658425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:23.659531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:23.660508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 613/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:23.665671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:23.765899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3399-7b25-4e13-9408-10629a2f7445 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.800051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.800656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.800940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.801215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:23.801935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:23.802243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:23.803743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:23.821419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 627/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Tue Sep 13 10:01:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:23.827571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:23.926405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d3d13-6db4-4278-8104-64f4e2a9d65a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:23.958550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:23.959473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:23.959781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:23.960065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:24.020449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:24.026786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:24.028270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:24.123905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 614/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 10:01:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 10:01:24.129893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:24.157193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:24.232949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07222bc8-52dd-4254-b9e9-c30dd7feaf00 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:24.267206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:24.267838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:24.268123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:24.268401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:24.278603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:24.280082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:24.295186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:24.296275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:24.297522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:24.297522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:24.297522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:24.297522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 628/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:24.302130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:24.394053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:24.399832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6e8b2-1041-4ec6-8eb1-cfec29c04f6e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:24.433873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:24.434972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:24.435406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:24.435682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:24.436316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:24.436617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:24.438008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:24.448932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 615/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:24.455272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:24.533158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:24.555316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0261048-63a6-40a1-bc8c-ead6296d2699 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:24.589115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:24.589743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:24.590021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:24.590856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:24.597162 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:24.598649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 10:01:24.610357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:24.611427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:24.612590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 629/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 10:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:24.617466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:24.716846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e0b39-55d0-4f98-bec6-b2951918fa18 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:24.750076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:24.750674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:24.750958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:24.751291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:24.751988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:24.752299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:24.753708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9203-980f-4f9f-8cdb-d9e8f3fa0fdb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:24.765395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 616/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 10:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:24.771628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:24.872739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97241-9d24-4910-adc6-9ff5d9969c12 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:24.906240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:24.906837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:24.907119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:24.907425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:24.908141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] RBAC: Adding query filter params (role.id=9a6b9c222c284a98b8c786285c749f03, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=751cbd1e77824b83aa3c9c03c637fadf) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:24.908447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:24.909870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3375ea3-e8c6-469d-a7d7-e2d70745f234 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:24.915785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 630/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 10:01:24 2022] GET /identity/v3/role_assignments?user.id=751cbd1e77824b83aa3c9c03c637fadf&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=9a6b9c222c284a98b8c786285c749f03 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:27.155414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a0e3e-00ec-4a07-a77a-ced40adc58ac None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:27.156003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a0e3e-00ec-4a07-a77a-ced40adc58ac None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:27.156003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a0e3e-00ec-4a07-a77a-ced40adc58ac None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:27.157391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 617/1247] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:27.162743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:27.163224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:27.163364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:27.170561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:27.182709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:27.188584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:27.250261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 631/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:27.528370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a0e3e-00ec-4a07-a77a-ced40adc58ac None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:27.547184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:27.627058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a0e3e-00ec-4a07-a77a-ced40adc58ac None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:27.659581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:27.660200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:27.660479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:27.660752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:27.671339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 10:01:27.676061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:27.678618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:27.679932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 618/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 10:01:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:27.683457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:27.787214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3665f5-91cd-4d8e-8b77-809f5b908ad6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:27.816551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:27.821295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:27.821890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:27.822166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:27.822436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:27.823085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:27.823416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:27.824809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:27.832648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 632/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 10:01:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:27.838447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:27.914904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:27.938257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc6b5c-c543-47e6-a71c-61452b87b29d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:27.971065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:27.971718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:27.972002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:27.972276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:27.981908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:27.983371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:27.998563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:28.004602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:28.005592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:28.005592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:28.005592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:28.005592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 619/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 10:01:28.005818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:28.110702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba1191-3091-46f5-9243-ae05a95cd0d8 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:28.143950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:28.144578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:28.144861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:28.145136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:28.145778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:28.146078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:28.147473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:28.159097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 633/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:28.165075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:28.263579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af657707-588d-4f30-b566-ae92eb651e4c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:28.288373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:28.296732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:28.297340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:28.300712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:28.300712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:28.304914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:28.306404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 10:01:28.317694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:28.319092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:28.320308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 620/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:28.326936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:28.386567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:28.431875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2da65-341c-4c02-a68f-7699d11b756d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:28.466231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:28.466832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:28.467128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:28.467429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:28.468147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:28.468453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:28.469871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:28.487255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 634/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Tue Sep 13 10:01:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:28.493202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:28.591247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea13c8c-064d-4afa-bd9e-040506e0a346 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:28.623860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:28.627644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:28.627644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:28.627644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:28.678646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:28.690362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:28.691835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:28.786987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 621/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 10:01:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 10:01:28.793058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:28.892924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc3e1e-ffd2-41c7-a2ce-be5fe03fd3c9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:28.906208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:28.927209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:28.927833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:28.928116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:28.928389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:28.938821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:28.940238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:28.955209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:28.960831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:28.961870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:28.961870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:28.961870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:28.961870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 635/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:28.962102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:29.046148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:29.057595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaba5d-dc13-4a79-9f5e-329bf89e9607 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:29.090161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:29.090757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:29.091036 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:29.091337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:29.091969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:29.092271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:29.093621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:29.104467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 622/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:01:29.110560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:29.211573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e4f93-fb52-4d07-9ea3-05c70f2e35c4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:29.244658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:29.245286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:29.245570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:29.245847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:29.253051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:29.254492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 10:01:29.265541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:29.272136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:29.273123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 636/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 10:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:29.273665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:29.376130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5157-7693-4eaa-96c0-17dadbef9e13 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:29.411724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:29.412333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:29.412617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:29.412896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:29.413613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:29.413922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:29.415689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d037203-52ac-4ac8-ba94-e15ce5b7aa20 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:29.428000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 623/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 10:01:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:29.434655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:29.462314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:29.538114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba802ea-050c-4c34-acd8-01e436f056f4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:29.572420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:29.573073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:29.573358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:29.573630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:29.574347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=751cbd1e77824b83aa3c9c03c637fadf) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:29.575241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:29.576669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a41fb-a612-49e8-9e3e-ef4580279e21 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:29.582306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 637/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 10:01:29 2022] GET /identity/v3/role_assignments?user.id=751cbd1e77824b83aa3c9c03c637fadf&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=435125566e954d20a3b8273b7c806b98 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:31.736099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765b4ac-fbb0-4d93-88ce-fd44016d02b6 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:31.736099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765b4ac-fbb0-4d93-88ce-fd44016d02b6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:31.736099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765b4ac-fbb0-4d93-88ce-fd44016d02b6 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:31.736099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 624/1261] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:31.740011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:31.740447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:31.740541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:31.758474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:31.759724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:31.825984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 638/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:32.099453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765b4ac-fbb0-4d93-88ce-fd44016d02b6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:32.110840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:32.199653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765b4ac-fbb0-4d93-88ce-fd44016d02b6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:32.232888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:32.233513 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:32.233796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:32.234071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:32.243824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 10:01:32.249313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:32.251107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:32.252828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 625/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 10:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:32.256538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:32.274439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:32.353160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ce06-362f-4d23-b9da-fd39179a88f4 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:32.386854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:32.387496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:32.387781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:32.388528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:32.388707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:32.389013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:32.390373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:32.398503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 639/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 10:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:32.404754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:32.472614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:32.506773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5c1a7-3c6b-4f10-b09d-7f9f554ecfe9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:32.539017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:32.540031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:32.540314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:32.540589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:32.550642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:32.552104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:32.567064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:32.568382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:32.569530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:32.569530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:32.569530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:32.569530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 626/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:32.574987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:32.655228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:32.674413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483f003-2771-44c4-bbe0-e6ecbe24f082 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:32.708991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:32.709599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:32.709882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:32.710166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:32.710801 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:32.711108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:32.712497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:32.724114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 640/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:32.729917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:32.829173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36956c2-34c1-4aef-832a-60000e177a6c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:32.847980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:32.862341 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:32.862950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:32.863255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:32.863520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:32.871290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:32.873182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 10:01:32.883902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:32.885021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:32.886158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 627/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:32.890852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:32.988835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a371bfc-c7e7-4630-b3c2-459bbed27742 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:33.023264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:33.023870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:33.024149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:33.024420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:33.025141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:33.025449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:33.026883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:33.035537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:33.044649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 641/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Tue Sep 13 10:01:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:01:33.050596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:33.149616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1e14e-d57e-4c2b-a605-dc3907c3c2eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:33.183127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:33.183779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:33.184055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:33.184322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:33.194397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:33.195859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:33.209539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:33.210634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:33.211623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:33.211623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:33.211623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:33.211623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 628/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:33.216999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:33.317318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3746ab-79e1-4077-857b-0abc3b53b56d None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:33.350905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:33.351537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:33.351821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:33.352099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:33.352735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:33.353042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:33.354400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:33.366093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 642/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:33.372563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:33.404398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:33.471591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaf340-2ed7-4877-a721-bffd2cd3f520 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:33.508984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:33.509609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:33.509889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:33.510162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:33.517292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:33.518894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 10:01:33.529949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:33.535690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:33.536837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 629/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 10:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:33.537447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:33.596341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:33.637077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d648d-1825-4ac3-a027-148f7be034de None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:33.670277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:33.670882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:33.671201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:33.671475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:33.672190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:33.672504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:33.673952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7f5f-5d10-4a8a-8529-c3d507ad11bc None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:33.686313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 643/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 10:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:33.692382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:33.788221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:33.799353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de46860-976e-4ad5-9523-0faa0b78098d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:33.832423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130dd54e-d1c6-423c-8950-7081f738794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:33.833233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130dd54e-d1c6-423c-8950-7081f738794f None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:33.833562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130dd54e-d1c6-423c-8950-7081f738794f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:33.833883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130dd54e-d1c6-423c-8950-7081f738794f None admin] PATH_INFO: `/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/751cbd1e77824b83aa3c9c03c637fadf/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:33.835966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130dd54e-d1c6-423c-8950-7081f738794f None admin] RBAC: Authorizing `identity:create_grant(project_id=cc4aca9e25c442aa9332d584d003a09a, user_id=751cbd1e77824b83aa3c9c03c637fadf, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:33.837605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130dd54e-d1c6-423c-8950-7081f738794f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:33.924567 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 630/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Tue Sep 13 10:01:33 2022] PUT /identity/v3/projects/cc4aca9e25c442aa9332d584d003a09a/users/751cbd1e77824b83aa3c9c03c637fadf/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:01:36.056705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155959-3eec-4aa1-87ca-a69d5f236fac None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:36.057232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155959-3eec-4aa1-87ca-a69d5f236fac None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:36.057232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155959-3eec-4aa1-87ca-a69d5f236fac None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:36.059264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 644/1274] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:36.064525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:36.064923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:36.065083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:36.083066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:36.084286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:36.150301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 631/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:36.424712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155959-3eec-4aa1-87ca-a69d5f236fac None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:36.526956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155959-3eec-4aa1-87ca-a69d5f236fac None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:36.544379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:36.562751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:36.563469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:36.563754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:36.564030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:36.574456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 10:01:36.579407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:36.580523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:36.581377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 645/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Tue Sep 13 10:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:36.586705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:36.670002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:36.687607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357e21-8329-4be4-bce4-d3b4c8f94c86 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:36.719833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:36.720505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:36.720791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:36.721046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:36.721728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:36.722031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:36.723462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:36.731942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 632/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Tue Sep 13 10:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:36.737868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:36.837178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbe137-9aac-4365-8d38-c98ffafbb178 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:36.872074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:36.872703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:36.872988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:36.873267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:36.883387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:36.884873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:36.899388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:36.900359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:36.901427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:36.901427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:36.901427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:36.901427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 646/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:36.906915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:37.016942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4b63d-1675-4d73-8be2-470002260710 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:37.053291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:37.053918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:37.054202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:37.054533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:37.055359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:37.055668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:37.057109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:37.071124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 633/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:37.077741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:37.110589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:37.188706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a7e2f-ca8d-406d-af72-5c283524f4f6 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:37.225371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:37.226063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:37.226348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:37.226648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:37.234883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:37.236408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 10:01:37.249960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:37.251089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:37.252142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 647/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:37.257160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:37.264947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:37.362724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53b7ab-2293-4b85-870a-48d2fedb4350 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:37.399961 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:37.400568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:37.400843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:37.401106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:37.401833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:37.402134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:37.404194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:37.421599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 634/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Tue Sep 13 10:01:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:37.427546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:37.516209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:37.525388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608501-6528-4a2c-8910-6970c17cf7b4 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:37.559511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:37.560122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:37.560383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:37.560653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:37.626708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:37.628111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:37.715450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:37.725630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 648/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Tue Sep 13 10:01:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 10:01:37.732175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:37.791361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:37.828292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85bb3-337b-4f49-b23b-e79b0613aa7d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:37.861500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:37.862115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:37.862397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:37.862698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:37.872867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:37.874290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 10:01:37.887982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:37.889093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 10:01:37.889988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 10:01:37.889988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 10:01:37.889988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:37.889988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 635/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:37.894824 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:37.993090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9ef5-3c9b-452f-97b0-f4a8af8e8934 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:38.026801 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:38.027425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:38.027710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:38.027984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:38.028617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:38.028920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:38.030268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:38.041533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 649/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Tue Sep 13 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:38.047660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:38.146353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629677a-b0dd-477c-8cc1-819c90f519d5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:38.159389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:38.179495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:38.180236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:38.180514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:38.180784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:38.187817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:38.189234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 10:01:38.200439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:38.201499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 13 10:01:38.202728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 636/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Tue Sep 13 10:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:38.207405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:38.257342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:38.305572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bfb26-02c1-4537-afcf-452767df4be2 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:38.337813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:38.338408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:38.338692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:38.338966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:38.339725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:38.340069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:38.347351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30498ea3-4c5e-4a9d-be9b-8ba0140b3239 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:38.353794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 650/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Tue Sep 13 10:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:38.360374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:38.460512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2c054-4ada-4dc7-b002-234d1a46a18e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:38.492286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:38.493899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:38.493899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:38.493899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:38.494213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] RBAC: Adding query filter params (role.id=435125566e954d20a3b8273b7c806b98, scope.project.id=cc4aca9e25c442aa9332d584d003a09a, user.id=751cbd1e77824b83aa3c9c03c637fadf) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:38.494526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:38.496476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fb24d-10bf-474b-b314-62651c1833ab None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:38.502014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 637/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Tue Sep 13 10:01:38 2022] GET /identity/v3/role_assignments?user.id=751cbd1e77824b83aa3c9c03c637fadf&scope.project.id=cc4aca9e25c442aa9332d584d003a09a&role.id=435125566e954d20a3b8273b7c806b98 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:40.690841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc2db4-0821-4372-848c-9edd9b50c191 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:40.691423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc2db4-0821-4372-848c-9edd9b50c191 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:40.691423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc2db4-0821-4372-848c-9edd9b50c191 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:40.692695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 651/1288] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:40.698050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:40.702428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:40.702428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:40.717569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:40.718781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:40.748968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:40.785121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 638/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:41.062095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc2db4-0821-4372-848c-9edd9b50c191 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:41.160417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc2db4-0821-4372-848c-9edd9b50c191 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:41.171578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:41.194120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:41.194735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:41.195017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:41.195313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:41.195881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:41.197226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:41.205747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:41.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:41.212550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 652/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 10:01:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:41.213605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:41.319648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f0c83-5732-461d-8cd5-ff9f717b8e52 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:41.356090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:41.356702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:41.357001 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:41.357287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:41.357867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:41.358167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:41.359542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80084ef-70c5-40c3-8395-2948eb34d9e7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:41.364237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 639/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Tue Sep 13 10:01:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:01:41.370126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:41.478860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c524c8-b894-4bae-a791-cb6201fa1fb9 None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:41.511208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:41.512431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:41.512710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:41.512980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] PATH_INFO: `/v3/services` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:41.513600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:41.513902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:41.515285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d74d08-c21c-481f-aee4-6853a23597a8 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:41.519619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 653/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Tue Sep 13 10:01:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:43.650945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658643-3765-44ae-8e85-4604734ed2cb None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:43.651490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658643-3765-44ae-8e85-4604734ed2cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:43.651490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658643-3765-44ae-8e85-4604734ed2cb None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:43.653435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 640/1293] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:43.658524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770b1b6-3064-4730-83c9-34b72a2af185 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:43.658925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770b1b6-3064-4730-83c9-34b72a2af185 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:43.659079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770b1b6-3064-4730-83c9-34b72a2af185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:43.677676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1770b1b6-3064-4730-83c9-34b72a2af185 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:43.678867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1770b1b6-3064-4730-83c9-34b72a2af185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:43.707446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:43.744640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 654/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:44.012221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01658643-3765-44ae-8e85-4604734ed2cb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:44.042412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:44.111394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01658643-3765-44ae-8e85-4604734ed2cb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:44.144784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4bb70-dff5-44bd-b101-da698bf286c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:44.145408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4bb70-dff5-44bd-b101-da698bf286c2 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:44.145691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4bb70-dff5-44bd-b101-da698bf286c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:44.145966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4bb70-dff5-44bd-b101-da698bf286c2 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:44.146602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4bb70-dff5-44bd-b101-da698bf286c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:44.147987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4bb70-dff5-44bd-b101-da698bf286c2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:44.238534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 641/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Tue Sep 13 10:01:44 2022] POST /identity/v3/services => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:01:46.402782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069443b-0767-4b54-b96d-cd46f51207c6 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:46.403239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069443b-0767-4b54-b96d-cd46f51207c6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:46.403382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069443b-0767-4b54-b96d-cd46f51207c6 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:46.404737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 655/1296] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:46.410004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:46.410423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:46.410544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:46.429349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:46.430521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:46.502241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 642/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:46.770183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069443b-0767-4b54-b96d-cd46f51207c6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:46.870796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069443b-0767-4b54-b96d-cd46f51207c6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:46.903462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:46.904094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:46.904374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:46.904691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] PATH_INFO: `/v3/services/placement` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:46.905267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:46.906629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:46.915591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:46.916575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:46.918102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 656/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 10:01:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:46.923596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:46.941987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:47.019862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40048d20-11a2-47fc-b686-aee06297f7ca None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:47.053760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:47.054288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:47.054569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:47.054842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:47.055495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:47.055799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:47.057145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:47.062728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 643/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Tue Sep 13 10:01:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:47.068721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:47.100629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:47.170335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454b2d5-bdb6-4453-9968-9d56bf5256fb None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:47.203674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:47.204275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:47.204553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:47.204823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:47.205509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] RBAC: Adding query filter params (interface=public, service_id=e93eb36173994c78a7b2361a2d5bb64f, region_id=RegionOne) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:47.205812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:47.207208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383feb-b7d3-453b-b41c-ebf73f5ce012 None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:47.212139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 657/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Tue Sep 13 10:01:47 2022] GET /identity/v3/endpoints?service_id=e93eb36173994c78a7b2361a2d5bb64f&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:47.218249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:47.300817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:47.318783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c42410-e982-4870-9749-a52f6c5a427a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:47.351366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ae098-21f2-4e89-9854-07557e738da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:47.351993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae098-21f2-4e89-9854-07557e738da9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:47.352277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae098-21f2-4e89-9854-07557e738da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:47.352552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae098-21f2-4e89-9854-07557e738da9 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:47.353153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ae098-21f2-4e89-9854-07557e738da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:47.354505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ae098-21f2-4e89-9854-07557e738da9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:47.362487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 644/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Tue Sep 13 10:01:47 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:01:49.624180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbddc34-1134-4e2e-9237-461a4afdcea3 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:49.624723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbddc34-1134-4e2e-9237-461a4afdcea3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:49.624723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbddc34-1134-4e2e-9237-461a4afdcea3 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:49.640646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 658/1302] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:01:49.640760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:49.640760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:49.640760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:49.650494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:01:49.652210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:01:49.716985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 645/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:01:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 10:01:49.980302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbddc34-1134-4e2e-9237-461a4afdcea3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:50.076498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:50.083316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbddc34-1134-4e2e-9237-461a4afdcea3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:50.116703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:50.117329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:50.117612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:50.117928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:50.118500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:50.119881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 10:01:50.128625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 10:01:50.129693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application with Lock( Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self._enter() Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application return self.creator() Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application created_value = creator( Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: ERROR keystone.server.flask.application Sep 13 10:01:50.130585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 659/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Tue Sep 13 10:01:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 10:01:50.136501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:50.212606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:01:50.235115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964a581-88f2-4a16-9e2b-71226846ceff None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:50.270297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:50.270965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:50.271274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:50.271544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:50.272166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:01:50.272467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:50.273926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891549a-639f-42d0-8b24-b81a3988ebc0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:50.279736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 646/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Tue Sep 13 10:01:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:01:50.285664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:01:50.385624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959b5b2-1847-4e89-9f28-b9b7fb0efc0f None admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:01:50.419485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a12a9a-5ccc-4b6b-9f08-9802bf6d26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:01:50.420091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12a9a-5ccc-4b6b-9f08-9802bf6d26ab None admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:01:50.420372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12a9a-5ccc-4b6b-9f08-9802bf6d26ab None admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:01:50.420647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12a9a-5ccc-4b6b-9f08-9802bf6d26ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:01:50.421312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a12a9a-5ccc-4b6b-9f08-9802bf6d26ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:01:50.422668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a12a9a-5ccc-4b6b-9f08-9802bf6d26ab None admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:01:50.494892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 660/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:01:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:02:00.275929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=30 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:02:00.490393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:24.081688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330f234-6320-45bb-a4d9-5565b21087c5 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:24.084882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330f234-6320-45bb-a4d9-5565b21087c5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:24.085896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330f234-6320-45bb-a4d9-5565b21087c5 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:24.092131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 647/1307] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:24.103428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e53e5-4857-4c1e-85f8-046fcfee33a2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:24.103428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e53e5-4857-4c1e-85f8-046fcfee33a2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:24.103428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e53e5-4857-4c1e-85f8-046fcfee33a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:24.230223 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c22e53e5-4857-4c1e-85f8-046fcfee33a2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:24.231880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c22e53e5-4857-4c1e-85f8-046fcfee33a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:24.354958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 661/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:24.661941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68026e35-4aab-43df-8946-ebf4e3f10617 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:24.662225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68026e35-4aab-43df-8946-ebf4e3f10617 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:24.662675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68026e35-4aab-43df-8946-ebf4e3f10617 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:24.664194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 648/1309] 10.222.0.62 () {54 vars in 965 bytes} [Tue Sep 13 10:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:24.671040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f541b24-ae83-4003-99b7-6b4a6851eb47 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:24.671363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f541b24-ae83-4003-99b7-6b4a6851eb47 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:24.671856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f541b24-ae83-4003-99b7-6b4a6851eb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:24.759455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1f541b24-ae83-4003-99b7-6b4a6851eb47 None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:24.760842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1f541b24-ae83-4003-99b7-6b4a6851eb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:24.784433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:24.851116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 662/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Tue Sep 13 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:24.858431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68026e35-4aab-43df-8946-ebf4e3f10617 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:24.860212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-68026e35-4aab-43df-8946-ebf4e3f10617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:24.978868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68026e35-4aab-43df-8946-ebf4e3f10617 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:25.059685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:25.060218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:25.060500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:25.060771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:25.062367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:25.120889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:25.178202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:25.180230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761eef74-1139-4987-ac79-cd44ced0d482 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:25.202997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 649/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:27.658879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05143f7-1751-4e06-b527-7b53aca89e75 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:27.659496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05143f7-1751-4e06-b527-7b53aca89e75 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:27.659496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05143f7-1751-4e06-b527-7b53aca89e75 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:27.661183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 663/1312] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:27.666438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902932-6346-4a9e-ac02-df4f9bbf7a9e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:27.666877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902932-6346-4a9e-ac02-df4f9bbf7a9e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:27.666979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902932-6346-4a9e-ac02-df4f9bbf7a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:27.747600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4c902932-6346-4a9e-ac02-df4f9bbf7a9e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:27.748819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4c902932-6346-4a9e-ac02-df4f9bbf7a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:27.842793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 650/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:28.127553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05143f7-1751-4e06-b527-7b53aca89e75 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:28.129150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a05143f7-1751-4e06-b527-7b53aca89e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:28.232670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05143f7-1751-4e06-b527-7b53aca89e75 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:28.304245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:28.306854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:28.307490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:28.307768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:28.308116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:28.309612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:28.424155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:28.425785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69737671-1199-43ef-acec-d2b0d32633a1 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:28.443738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 664/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:31.085435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315741-f78e-4437-b0d3-f376094f945d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:31.085998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315741-f78e-4437-b0d3-f376094f945d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:31.085998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315741-f78e-4437-b0d3-f376094f945d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:31.087370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 651/1315] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:31.092492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7341e-e3b1-41e6-82d8-58dde92b337d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:31.092915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7341e-e3b1-41e6-82d8-58dde92b337d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:31.093086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7341e-e3b1-41e6-82d8-58dde92b337d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:31.158118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-12a7341e-e3b1-41e6-82d8-58dde92b337d None None] MFA Rules not processed for user `a736dfa50d3841bb971419573615ad37`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:31.159385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-12a7341e-e3b1-41e6-82d8-58dde92b337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:31.215819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:31.260640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 665/1316] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:31.547586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315741-f78e-4437-b0d3-f376094f945d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:31.549124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6e315741-f78e-4437-b0d3-f376094f945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:31.560477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:31.650815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315741-f78e-4437-b0d3-f376094f945d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:31.721860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:31.722468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:31.722751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:31.723027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:31.724659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:31.787144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:31.832893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:31.834412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030ccf3-2c5e-4f3f-93a5-65a16e1f2f31 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:31.852919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 652/1317] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:34.679822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f6b8-5010-4af2-a6f2-d001842f7433 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:34.680326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f6b8-5010-4af2-a6f2-d001842f7433 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:34.680326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f6b8-5010-4af2-a6f2-d001842f7433 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:34.681643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 666/1318] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:34.686742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0def4-4292-4876-8f64-cc97793be7c5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:34.687141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0def4-4292-4876-8f64-cc97793be7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:34.687329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0def4-4292-4876-8f64-cc97793be7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:34.751389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8ad0def4-4292-4876-8f64-cc97793be7c5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:34.752605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8ad0def4-4292-4876-8f64-cc97793be7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:34.846883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 653/1319] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:35.131900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f6b8-5010-4af2-a6f2-d001842f7433 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:35.133532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fb53f6b8-5010-4af2-a6f2-d001842f7433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:35.241259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f6b8-5010-4af2-a6f2-d001842f7433 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:35.312111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:35.312710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:35.312985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:35.313253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:35.314831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:35.321988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:35.422549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:35.424088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eded79-f129-4d27-85a5-a1553a178e5f service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:35.441593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 667/1320] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:38.463903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c697ab-9e75-46f6-a663-be3e9526e888 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:38.464452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c697ab-9e75-46f6-a663-be3e9526e888 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:38.464452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c697ab-9e75-46f6-a663-be3e9526e888 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:38.465830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 654/1321] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:38.471112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e7dc-e929-4b71-bc04-3aab7e984aaf None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:38.471543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e7dc-e929-4b71-bc04-3aab7e984aaf None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:38.471700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e7dc-e929-4b71-bc04-3aab7e984aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:38.538152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-10b3e7dc-e929-4b71-bc04-3aab7e984aaf None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:38.539385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-10b3e7dc-e929-4b71-bc04-3aab7e984aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:38.598819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:38.639549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 668/1322] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:38.923507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c697ab-9e75-46f6-a663-be3e9526e888 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:38.925075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e4c697ab-9e75-46f6-a663-be3e9526e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:38.945676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:39.030228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c697ab-9e75-46f6-a663-be3e9526e888 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:39.105133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:39.105667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:39.105949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:39.106220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:39.107865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:39.180940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:39.219625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:39.221067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8e8bf-f4e6-4c72-8796-e7a37b7748a1 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:39.240077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 655/1323] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:41.469962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2b3d0-ed49-43e2-81e7-0a06a1c5033d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:41.470499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2b3d0-ed49-43e2-81e7-0a06a1c5033d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:41.470499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2b3d0-ed49-43e2-81e7-0a06a1c5033d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:41.471920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 669/1324] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:41.477219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e554b-8d21-4176-a95e-631a2f610061 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:41.477603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e554b-8d21-4176-a95e-631a2f610061 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:41.477781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e554b-8d21-4176-a95e-631a2f610061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:41.550241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dc8e554b-8d21-4176-a95e-631a2f610061 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:41.551484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dc8e554b-8d21-4176-a95e-631a2f610061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:41.651780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 656/1325] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:41.940211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2b3d0-ed49-43e2-81e7-0a06a1c5033d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:41.941847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1cf2b3d0-ed49-43e2-81e7-0a06a1c5033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:42.049083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2b3d0-ed49-43e2-81e7-0a06a1c5033d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:42.124264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:42.124993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:42.125282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:42.125558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:42.127244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:42.144464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:42.238034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:42.239532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279085bc-acd6-4d4e-91b2-52a1779d489e service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:42.258328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 670/1326] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:44.488096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490cd1-c543-405f-8b38-08b1302a400b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:44.488602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490cd1-c543-405f-8b38-08b1302a400b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:44.488694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490cd1-c543-405f-8b38-08b1302a400b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:44.489997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 657/1327] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:44.495180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd878f-077d-42c8-b5e0-030dccd54952 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:44.495545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd878f-077d-42c8-b5e0-030dccd54952 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:44.495757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd878f-077d-42c8-b5e0-030dccd54952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:44.563643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8bdd878f-077d-42c8-b5e0-030dccd54952 None None] MFA Rules not processed for user `a736dfa50d3841bb971419573615ad37`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:44.564776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8bdd878f-077d-42c8-b5e0-030dccd54952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:44.629829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:44.661938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 671/1328] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:44.956276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7bead-325f-4321-afe3-a0ad303a6a93 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:44.956565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7bead-325f-4321-afe3-a0ad303a6a93 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:44.956837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7bead-325f-4321-afe3-a0ad303a6a93 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:44.958195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 658/1329] 10.222.0.62 () {54 vars in 965 bytes} [Tue Sep 13 10:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:44.964498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041af4a-253a-47b4-b65d-70c50758c515 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:44.964866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041af4a-253a-47b4-b65d-70c50758c515 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:44.965215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041af4a-253a-47b4-b65d-70c50758c515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:45.046929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a041af4a-253a-47b4-b65d-70c50758c515 None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:45.048219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a041af4a-253a-47b4-b65d-70c50758c515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:45.127549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 672/1330] 10.222.0.62 () {58 vars in 1062 bytes} [Tue Sep 13 10:07:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:45.136375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7bead-325f-4321-afe3-a0ad303a6a93 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:45.137994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2cd7bead-325f-4321-afe3-a0ad303a6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:45.171592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:45.243244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7bead-325f-4321-afe3-a0ad303a6a93 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:45.316290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:45.316890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:45.317168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:45.317436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:45.319043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:45.399370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:45.431460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:45.432920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d055aa-2a47-4d4f-8360-e83d1de1b7e2 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:45.457452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 659/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:48.463527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42bb7b-5ac9-44d6-8a72-322ec37d8254 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:48.464102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42bb7b-5ac9-44d6-8a72-322ec37d8254 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:48.464102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42bb7b-5ac9-44d6-8a72-322ec37d8254 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:48.465443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 673/1332] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:48.470691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e45919-db77-44ef-b8c1-bd91277e6dc1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:48.471089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e45919-db77-44ef-b8c1-bd91277e6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:48.471276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e45919-db77-44ef-b8c1-bd91277e6dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:48.535936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-89e45919-db77-44ef-b8c1-bd91277e6dc1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:48.538771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-89e45919-db77-44ef-b8c1-bd91277e6dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:48.632920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 660/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:48.916459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42bb7b-5ac9-44d6-8a72-322ec37d8254 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:48.918050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fd42bb7b-5ac9-44d6-8a72-322ec37d8254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:48.987464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:49.020508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42bb7b-5ac9-44d6-8a72-322ec37d8254 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:49.093363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:49.093968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:49.094250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:49.094524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:49.096216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:49.203411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:49.204774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10aef4-f1fd-4faa-b6e3-53b4bf90c073 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:49.214640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:49.222692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 674/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:51.421783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ca8c6-821f-43c6-9968-c87214b76840 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:51.422245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ca8c6-821f-43c6-9968-c87214b76840 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:51.422352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ca8c6-821f-43c6-9968-c87214b76840 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:51.423722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 661/1335] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:51.429607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08c68-f04a-4587-a14f-e0bb56664554 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:51.429996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08c68-f04a-4587-a14f-e0bb56664554 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:51.430165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08c68-f04a-4587-a14f-e0bb56664554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:51.495974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bfb08c68-f04a-4587-a14f-e0bb56664554 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:51.497193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bfb08c68-f04a-4587-a14f-e0bb56664554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:51.593895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 675/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:07:51.878082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ca8c6-821f-43c6-9968-c87214b76840 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:51.879674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3c5ca8c6-821f-43c6-9968-c87214b76840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:51.920013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:51.983326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ca8c6-821f-43c6-9968-c87214b76840 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:52.057248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:52.057843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:52.058123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:52.058394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:52.060079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:52.144248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:52.170841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:52.172311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73280e03-282c-4228-b6c7-7ce99f68c230 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:52.190480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 662/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:07:55.071483 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cdc37-8b74-455c-aada-9d7d43916e43 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:55.072043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cdc37-8b74-455c-aada-9d7d43916e43 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:55.072043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cdc37-8b74-455c-aada-9d7d43916e43 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:55.073417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 676/1338] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:07:55.078511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc732b-caf4-47c1-988d-01a8487eaf10 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:55.079024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc732b-caf4-47c1-988d-01a8487eaf10 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:55.079121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc732b-caf4-47c1-988d-01a8487eaf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:55.145767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-22dc732b-caf4-47c1-988d-01a8487eaf10 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:07:55.147549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-22dc732b-caf4-47c1-988d-01a8487eaf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:55.245519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 663/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:07:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:07:55.530703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cdc37-8b74-455c-aada-9d7d43916e43 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:07:55.532284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-247cdc37-8b74-455c-aada-9d7d43916e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:55.609897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:07:55.637437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cdc37-8b74-455c-aada-9d7d43916e43 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:07:55.712762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:07:55.713376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:07:55.713658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:07:55.713934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:07:55.715639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:07:55.824404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:07:55.825826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7e0b-133c-4fc5-89dd-722f6500bf5b service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:07:55.844390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 677/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:07:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:00.114012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4c9a2-4869-4cd8-83f0-630d2e2fa89b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:00.115088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4c9a2-4869-4cd8-83f0-630d2e2fa89b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:00.115088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4c9a2-4869-4cd8-83f0-630d2e2fa89b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:00.116640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 664/1341] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:00.121888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d466-1365-48c0-b398-4b7f1929449f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:00.122303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d466-1365-48c0-b398-4b7f1929449f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:00.122445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d466-1365-48c0-b398-4b7f1929449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:00.127960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:00.190649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e1f3d466-1365-48c0-b398-4b7f1929449f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:00.191912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e1f3d466-1365-48c0-b398-4b7f1929449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:00.293643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 678/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:00.575514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4c9a2-4869-4cd8-83f0-630d2e2fa89b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:00.577203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a1e4c9a2-4869-4cd8-83f0-630d2e2fa89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:00.626928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:00.683377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4c9a2-4869-4cd8-83f0-630d2e2fa89b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:00.756479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:00.757094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:00.757375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:00.757649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:00.759270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:00.855457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:00.871454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:00.872917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd2bdc-4fe2-4a43-8b4f-4c22b1ce3a50 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:00.891848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 665/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:04.032139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6906b91-0e7c-4b05-a14c-fd3f85c4b15d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:04.032706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6906b91-0e7c-4b05-a14c-fd3f85c4b15d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:04.032706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6906b91-0e7c-4b05-a14c-fd3f85c4b15d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:04.034039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 679/1344] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:04.039414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b09c9-0fdd-42c0-9baa-b7d9eb7e0857 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:04.039826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b09c9-0fdd-42c0-9baa-b7d9eb7e0857 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:04.039973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b09c9-0fdd-42c0-9baa-b7d9eb7e0857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:04.111651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8b9b09c9-0fdd-42c0-9baa-b7d9eb7e0857 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:04.112873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8b9b09c9-0fdd-42c0-9baa-b7d9eb7e0857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:04.207995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 666/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:04.491210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6906b91-0e7c-4b05-a14c-fd3f85c4b15d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:04.492773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a6906b91-0e7c-4b05-a14c-fd3f85c4b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:04.582756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:04.600913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6906b91-0e7c-4b05-a14c-fd3f85c4b15d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:04.680069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:04.680683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:04.681072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:04.681354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:04.682979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:04.822296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:04.823791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6965dc1-e6aa-4fff-a759-4c7fd9ff39e2 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:04.844120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 680/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:09.360585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886eef3-31b3-4071-978d-39c955ace9e2 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:09.361093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886eef3-31b3-4071-978d-39c955ace9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:09.361193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886eef3-31b3-4071-978d-39c955ace9e2 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:09.362494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 667/1347] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:09.368683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c80e-8417-40e8-a161-feaabdb32d2a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:09.369130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c80e-8417-40e8-a161-feaabdb32d2a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:09.369229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c80e-8417-40e8-a161-feaabdb32d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:09.392903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:09.443365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f0f5c80e-8417-40e8-a161-feaabdb32d2a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:09.444604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f0f5c80e-8417-40e8-a161-feaabdb32d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:09.557046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 681/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:09.840955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886eef3-31b3-4071-978d-39c955ace9e2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:09.842473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2886eef3-31b3-4071-978d-39c955ace9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:09.900434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:09.946573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886eef3-31b3-4071-978d-39c955ace9e2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:10.018754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:10.019365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:10.019646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:10.019916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:10.021566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:10.128317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:10.131816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:10.133235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e455617-b71b-4de4-91ff-b792084bff21 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:10.151236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 668/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:15.095932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90616c-e070-4e50-bd93-018da3c795f1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:15.096835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90616c-e070-4e50-bd93-018da3c795f1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:15.097341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90616c-e070-4e50-bd93-018da3c795f1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:15.102736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 682/1350] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:08:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:15.112212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99ad3-54a9-4411-9759-8aa1b976c380 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:15.113162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99ad3-54a9-4411-9759-8aa1b976c380 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:15.113598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99ad3-54a9-4411-9759-8aa1b976c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:15.257612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c9e99ad3-54a9-4411-9759-8aa1b976c380 None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:15.259080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c9e99ad3-54a9-4411-9759-8aa1b976c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:15.401223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 669/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:08:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:19.554291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=59 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:19.666329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb5d1d-9b67-4f28-83ae-6fe4b777653f None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:19.666858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb5d1d-9b67-4f28-83ae-6fe4b777653f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:19.667408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb5d1d-9b67-4f28-83ae-6fe4b777653f None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:19.669653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 683/1352] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:08:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:19.674965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e390-8801-4f0b-9784-0770483457ef None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:19.675489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e390-8801-4f0b-9784-0770483457ef None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:19.675657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e390-8801-4f0b-9784-0770483457ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:19.731198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:19.754997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5f75e390-8801-4f0b-9784-0770483457ef None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:19.756753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5f75e390-8801-4f0b-9784-0770483457ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:19.877670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 670/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:08:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:21.023220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0285ea6-14e9-4512-9acd-7cc1fd0157a1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:21.023710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0285ea6-14e9-4512-9acd-7cc1fd0157a1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:21.023710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0285ea6-14e9-4512-9acd-7cc1fd0157a1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:21.025577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 684/1354] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:21.030670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c403ac-7780-4d66-9b8b-5aa1510f4b69 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:21.031073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c403ac-7780-4d66-9b8b-5aa1510f4b69 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:21.031256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c403ac-7780-4d66-9b8b-5aa1510f4b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:21.109040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-23c403ac-7780-4d66-9b8b-5aa1510f4b69 None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:21.110241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-23c403ac-7780-4d66-9b8b-5aa1510f4b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:21.153666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:21.220621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 671/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:08:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:08:24.911973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d3bb0-d98d-48f2-9229-710675b6aa51 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:24.912515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d3bb0-d98d-48f2-9229-710675b6aa51 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:24.912515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d3bb0-d98d-48f2-9229-710675b6aa51 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:24.914072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 685/1356] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:24.919261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d13364-703b-49ff-9480-b36f34d6ad6f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:24.919726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d13364-703b-49ff-9480-b36f34d6ad6f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:24.919831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d13364-703b-49ff-9480-b36f34d6ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:24.989715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f8d13364-703b-49ff-9480-b36f34d6ad6f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:24.990899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f8d13364-703b-49ff-9480-b36f34d6ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:25.090262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 672/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:25.388616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a200994-671b-458b-9f94-64820708a04f None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:25.388910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a200994-671b-458b-9f94-64820708a04f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:25.389189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a200994-671b-458b-9f94-64820708a04f None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:25.390816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 686/1358] 10.222.0.62 () {54 vars in 962 bytes} [Tue Sep 13 10:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:25.408057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2800d7-079a-49c4-aad1-281d93df3e70 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:25.415368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2800d7-079a-49c4-aad1-281d93df3e70 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:25.415368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2800d7-079a-49c4-aad1-281d93df3e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:25.426631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:25.635780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6f2800d7-079a-49c4-aad1-281d93df3e70 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:25.638872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6f2800d7-079a-49c4-aad1-281d93df3e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:25.788588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 673/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Tue Sep 13 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:25.796212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a200994-671b-458b-9f94-64820708a04f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:25.797955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9a200994-671b-458b-9f94-64820708a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:25.978142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a200994-671b-458b-9f94-64820708a04f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:26.065548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:26.108281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:26.108897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:26.109174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:26.109441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:26.111138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:26.207793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:26.209296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564c361-0ca4-4991-ba7c-ed4cdc6dca22 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:26.228914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 687/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:28.636386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaed343-c107-4d8e-975e-8b8d3e0fe240 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:28.636894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaed343-c107-4d8e-975e-8b8d3e0fe240 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:28.636974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaed343-c107-4d8e-975e-8b8d3e0fe240 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:28.638257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 674/1361] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:28.643780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc4872-167d-4642-9701-23d4480221c6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:28.648394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc4872-167d-4642-9701-23d4480221c6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:28.648394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc4872-167d-4642-9701-23d4480221c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:28.714237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2bcc4872-167d-4642-9701-23d4480221c6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:28.715399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2bcc4872-167d-4642-9701-23d4480221c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:28.756315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:28.819474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 688/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:29.110108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaed343-c107-4d8e-975e-8b8d3e0fe240 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:29.112046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-abaed343-c107-4d8e-975e-8b8d3e0fe240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:29.198488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:29.303891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaed343-c107-4d8e-975e-8b8d3e0fe240 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:29.427327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:29.428338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:29.428840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:29.429337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:29.432359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:29.458811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:29.534148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:29.536423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ad9a7-a13a-47aa-a8d2-58c579b4c0f2 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:29.556206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 675/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:32.187346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0731c-b7e3-46bf-beba-5f3d4193336d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:32.187920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0731c-b7e3-46bf-beba-5f3d4193336d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:32.187920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0731c-b7e3-46bf-beba-5f3d4193336d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:32.189417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 689/1364] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:32.194529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9fdff-3ae5-4953-abc1-628092dbfdab None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:32.194969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9fdff-3ae5-4953-abc1-628092dbfdab None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:32.195140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9fdff-3ae5-4953-abc1-628092dbfdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:32.268128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8ce9fdff-3ae5-4953-abc1-628092dbfdab None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:32.270029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8ce9fdff-3ae5-4953-abc1-628092dbfdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:32.374282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:32.383202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 676/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:32.669587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69d4ab-4a46-4915-ba64-9af58573062e None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:32.669916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69d4ab-4a46-4915-ba64-9af58573062e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:32.670140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69d4ab-4a46-4915-ba64-9af58573062e None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:32.671725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 690/1366] 10.222.0.62 () {54 vars in 962 bytes} [Tue Sep 13 10:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:32.677850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd64f-6f70-4d8e-9a5a-917f8e3723ee None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:32.678208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd64f-6f70-4d8e-9a5a-917f8e3723ee None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:32.678558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd64f-6f70-4d8e-9a5a-917f8e3723ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:32.763991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b0ebd64f-6f70-4d8e-9a5a-917f8e3723ee None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:32.765215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b0ebd64f-6f70-4d8e-9a5a-917f8e3723ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:32.891627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 677/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Tue Sep 13 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:32.899884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69d4ab-4a46-4915-ba64-9af58573062e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:32.901399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ab69d4ab-4a46-4915-ba64-9af58573062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:33.061459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69d4ab-4a46-4915-ba64-9af58573062e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:33.082707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:33.086731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f4827-19bc-4293-884c-869b0f22826c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:33.087011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f4827-19bc-4293-884c-869b0f22826c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:33.087319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f4827-19bc-4293-884c-869b0f22826c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:33.089092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 678/1368] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 10:08:33.093919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc9fc6-95ff-41f0-8e15-52353d0535e7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:33.094203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc9fc6-95ff-41f0-8e15-52353d0535e7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:33.094478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc9fc6-95ff-41f0-8e15-52353d0535e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:33.163661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:33.172529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-49dc9fc6-95ff-41f0-8e15-52353d0535e7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:33.173830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-49dc9fc6-95ff-41f0-8e15-52353d0535e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:33.190230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:33.191055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:33.191263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:33.191476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:33.193243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:33.295024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:33.296699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 679/1369] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:08:33.297142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37763-88d5-4ef0-81eb-8296162dba2e service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:33.319419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 691/1370] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:33.599485 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f656c2-5d07-4011-a8b1-368dbf5444ec None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:33.599812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f656c2-5d07-4011-a8b1-368dbf5444ec None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:33.600103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f656c2-5d07-4011-a8b1-368dbf5444ec None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:33.602568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 680/1371] 10.222.0.62 () {54 vars in 964 bytes} [Tue Sep 13 10:08:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:33.608314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d953bdf-9639-4e40-92b8-c41ea2ce097a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:33.608790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d953bdf-9639-4e40-92b8-c41ea2ce097a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:33.609078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d953bdf-9639-4e40-92b8-c41ea2ce097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:33.643655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:33.697814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0d953bdf-9639-4e40-92b8-c41ea2ce097a None None] MFA Rules not processed for user `566f088827ab4710bf54d64415831a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:33.699106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0d953bdf-9639-4e40-92b8-c41ea2ce097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:33.786002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 692/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Tue Sep 13 10:08:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:33.793669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f656c2-5d07-4011-a8b1-368dbf5444ec None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:33.795212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-35f656c2-5d07-4011-a8b1-368dbf5444ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:33.904568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f656c2-5d07-4011-a8b1-368dbf5444ec None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:33.935738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:33.981438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:33.982042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:33.982318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:33.982587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:33.985093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:34.103939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:34.105379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918b841-0cc9-4ff5-bf09-5bb354bb2319 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:34.125089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 681/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:35.737756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9940-737c-4874-a53b-451646a7fc12 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:35.738633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9940-737c-4874-a53b-451646a7fc12 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:35.738915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9940-737c-4874-a53b-451646a7fc12 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:35.740403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 693/1374] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:35.746536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be375-1e6d-4b81-abea-f73e59da455d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:35.747111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be375-1e6d-4b81-abea-f73e59da455d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:35.747111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be375-1e6d-4b81-abea-f73e59da455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:35.825346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c18be375-1e6d-4b81-abea-f73e59da455d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:35.827686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c18be375-1e6d-4b81-abea-f73e59da455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:35.836282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:35.934374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 682/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:36.234365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9940-737c-4874-a53b-451646a7fc12 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:36.236299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e4de9940-737c-4874-a53b-451646a7fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:36.361224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:36.400536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9940-737c-4874-a53b-451646a7fc12 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:36.528399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:36.529020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:36.529298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:36.529574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:36.531214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:36.611663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:36.624738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:36.626380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d61b0f-b9db-480c-bafd-7a7c5b08b510 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:36.645421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 694/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:39.026934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3681bc-da5e-4493-8af4-2ac3a0fbd356 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:39.027504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3681bc-da5e-4493-8af4-2ac3a0fbd356 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:39.027504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3681bc-da5e-4493-8af4-2ac3a0fbd356 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:39.029107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 683/1377] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:39.034417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812141ac-6894-4439-aa54-e7cc928486e6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:39.034807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812141ac-6894-4439-aa54-e7cc928486e6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:39.035031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812141ac-6894-4439-aa54-e7cc928486e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:39.102201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-812141ac-6894-4439-aa54-e7cc928486e6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:39.103507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-812141ac-6894-4439-aa54-e7cc928486e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:39.208156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 695/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:39.507892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3681bc-da5e-4493-8af4-2ac3a0fbd356 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:39.509552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ad3681bc-da5e-4493-8af4-2ac3a0fbd356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:39.656819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:39.667357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3681bc-da5e-4493-8af4-2ac3a0fbd356 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:39.787445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:39.788055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:39.788339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:39.788606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:39.790235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:39.895971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:39.897483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeeb379-b64b-4d8e-a386-5fd43f48ccd5 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:39.908491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:39.917888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 684/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:42.289655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a70e-19a8-4b3a-bc07-f40d80301d3c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:42.290210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a70e-19a8-4b3a-bc07-f40d80301d3c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:42.290210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a70e-19a8-4b3a-bc07-f40d80301d3c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:42.291744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 696/1380] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:42.297146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3cf69-85bb-4367-9167-6c0d72b78dde None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:42.297561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3cf69-85bb-4367-9167-6c0d72b78dde None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:42.297775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3cf69-85bb-4367-9167-6c0d72b78dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:42.366136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bab3cf69-85bb-4367-9167-6c0d72b78dde None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:42.367486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bab3cf69-85bb-4367-9167-6c0d72b78dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:42.471542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 685/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:42.759870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216a70e-19a8-4b3a-bc07-f40d80301d3c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:42.762020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e216a70e-19a8-4b3a-bc07-f40d80301d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:42.842694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:42.940946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216a70e-19a8-4b3a-bc07-f40d80301d3c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:43.092370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:43.093063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:43.093375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:43.093718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:43.095983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:43.122314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:43.205904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:43.208230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575f84a-758c-4fa9-ae5c-dd52e0312095 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:43.229669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 697/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:45.871145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fe2c6-28cf-4cac-9481-10d4256d8f6b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:45.871694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fe2c6-28cf-4cac-9481-10d4256d8f6b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:45.871878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fe2c6-28cf-4cac-9481-10d4256d8f6b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:45.873292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 686/1383] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 10:08:45.878737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9fac0-599b-40f8-a730-6e12a0e975ae None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:45.879216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9fac0-599b-40f8-a730-6e12a0e975ae None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:45.879325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9fac0-599b-40f8-a730-6e12a0e975ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:45.948250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-cae9fac0-599b-40f8-a730-6e12a0e975ae None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:45.949479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cae9fac0-599b-40f8-a730-6e12a0e975ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:46.041716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:46.047471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 698/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:46.333277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fe2c6-28cf-4cac-9481-10d4256d8f6b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:46.334814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-372fe2c6-28cf-4cac-9481-10d4256d8f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:46.418795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:46.491469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fe2c6-28cf-4cac-9481-10d4256d8f6b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:46.615835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:46.616403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:46.616685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:46.616954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:46.618569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:46.658400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:46.720912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:46.722697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427477a-bdac-4896-b4f2-25155621e033 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:46.741079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 687/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:49.069679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca04b52-3381-42c7-8032-72fe0924a4e6 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:49.070222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca04b52-3381-42c7-8032-72fe0924a4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:49.070222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca04b52-3381-42c7-8032-72fe0924a4e6 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:49.071650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 699/1386] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:49.077065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcebb8-20ef-4752-a18f-99ad99d00e80 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:49.077502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcebb8-20ef-4752-a18f-99ad99d00e80 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:49.077649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcebb8-20ef-4752-a18f-99ad99d00e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:49.145790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-aebcebb8-20ef-4752-a18f-99ad99d00e80 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:49.147007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aebcebb8-20ef-4752-a18f-99ad99d00e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:49.245213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 688/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:49.538304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca04b52-3381-42c7-8032-72fe0924a4e6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:49.539866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3ca04b52-3381-42c7-8032-72fe0924a4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:49.599490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc189f-6b07-4c1a-8bf3-dd71481cac9b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:49.599773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc189f-6b07-4c1a-8bf3-dd71481cac9b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:49.600049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc189f-6b07-4c1a-8bf3-dd71481cac9b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:49.601412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 689/1388] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:49.606317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac79d3-94b1-419c-89a6-54a93d31b923 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:49.606600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac79d3-94b1-419c-89a6-54a93d31b923 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:49.606886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac79d3-94b1-419c-89a6-54a93d31b923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:49.621853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:49.674162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9eac79d3-94b1-419c-89a6-54a93d31b923 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:49.675365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9eac79d3-94b1-419c-89a6-54a93d31b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:49.695696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca04b52-3381-42c7-8032-72fe0924a4e6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:49.765429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:49.774799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 690/1389] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:08:49.812616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:49.813240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:49.813521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:49.813809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:49.816034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:49.909210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:49.910813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cf742-fe40-455a-b945-8145da55a676 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:49.929695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 700/1390] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:50.088390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adec6-6f0f-4e6f-8ecb-ed8ac77e45b9 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:50.088903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adec6-6f0f-4e6f-8ecb-ed8ac77e45b9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:50.089006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adec6-6f0f-4e6f-8ecb-ed8ac77e45b9 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:50.090361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 691/1391] 10.222.0.62 () {54 vars in 966 bytes} [Tue Sep 13 10:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:50.095794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c372cc-e1f2-4380-b6d0-3e8b3d6eeaff None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:50.096231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c372cc-e1f2-4380-b6d0-3e8b3d6eeaff None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:50.096395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c372cc-e1f2-4380-b6d0-3e8b3d6eeaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:50.181461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-79c372cc-e1f2-4380-b6d0-3e8b3d6eeaff None None] MFA Rules not processed for user `5950076b981b43feba3185f1fb513729`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:50.182852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-79c372cc-e1f2-4380-b6d0-3e8b3d6eeaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:50.207117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:50.269488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 701/1392] 10.222.0.62 () {58 vars in 1063 bytes} [Tue Sep 13 10:08:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:08:50.278644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546adec6-6f0f-4e6f-8ecb-ed8ac77e45b9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:50.280220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-546adec6-6f0f-4e6f-8ecb-ed8ac77e45b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:50.378446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:50.385102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546adec6-6f0f-4e6f-8ecb-ed8ac77e45b9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:50.465827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:50.466361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:50.466640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:50.466912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:50.468648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:50.592306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:50.593772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcaf6bc-e414-4886-a57c-d2e140a2bc28 service glance] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:50.613846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 692/1393] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:08:51.482776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a37c58-220a-4d50-877d-1e0bed15b16c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:51.483252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a37c58-220a-4d50-877d-1e0bed15b16c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:51.483391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a37c58-220a-4d50-877d-1e0bed15b16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:51.553122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-86a37c58-220a-4d50-877d-1e0bed15b16c None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:51.554389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-86a37c58-220a-4d50-877d-1e0bed15b16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:51.641861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 702/1394] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:51.684986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989bc87-980f-4c9a-9991-383fa96d0df2 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:51.685494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989bc87-980f-4c9a-9991-383fa96d0df2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:51.685611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989bc87-980f-4c9a-9991-383fa96d0df2 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:51.686930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 693/1395] 10.222.0.62 () {54 vars in 943 bytes} [Tue Sep 13 10:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:51.694377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5e3b-6db7-48d5-931e-db3a53bd3bab None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:51.694663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5e3b-6db7-48d5-931e-db3a53bd3bab None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:51.694964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5e3b-6db7-48d5-931e-db3a53bd3bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:51.742458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:51.772476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2edd5e3b-6db7-48d5-931e-db3a53bd3bab None None] MFA Rules not processed for user `174cca9647954ad1afdd4e6b8fba18f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:51.773689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2edd5e3b-6db7-48d5-931e-db3a53bd3bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:51.878748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 703/1396] 10.222.0.62 () {58 vars in 1040 bytes} [Tue Sep 13 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:08:51.888155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989bc87-980f-4c9a-9991-383fa96d0df2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:51.889693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b989bc87-980f-4c9a-9991-383fa96d0df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:51.913609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:52.025338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989bc87-980f-4c9a-9991-383fa96d0df2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:52.128374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:52.128982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:52.129263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:52.129582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:52.131386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:52.138599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:52.218559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:52.220081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ae521-1bff-4970-9ed3-36ab8d67d8fe service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:52.228968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 694/1397] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:08:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:08:52.605239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39006760-a111-4f88-a381-e1bc2d5162d7 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:52.605903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39006760-a111-4f88-a381-e1bc2d5162d7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:52.605903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39006760-a111-4f88-a381-e1bc2d5162d7 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:52.607226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 704/1398] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:52.613325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c2513-c0e0-4792-b68c-2b7ef95ecd0e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:52.613618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c2513-c0e0-4792-b68c-2b7ef95ecd0e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:52.614018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c2513-c0e0-4792-b68c-2b7ef95ecd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:52.689629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5d7c2513-c0e0-4792-b68c-2b7ef95ecd0e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:52.690906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5d7c2513-c0e0-4792-b68c-2b7ef95ecd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:52.809436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:52.811752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 695/1399] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:53.104547 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39006760-a111-4f88-a381-e1bc2d5162d7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:53.106083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-39006760-a111-4f88-a381-e1bc2d5162d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:53.218228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:53.281391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39006760-a111-4f88-a381-e1bc2d5162d7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:53.422208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:53.422953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:53.423265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:53.423573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:53.425764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:53.497901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:53.534521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:53.535751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84625da5-d400-4b94-bf7f-c9dd476fb20d service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:53.559140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 705/1400] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:55.923737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd43cf-65ef-490b-896b-c63f4af2e659 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:55.924452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd43cf-65ef-490b-896b-c63f4af2e659 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:55.924452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd43cf-65ef-490b-896b-c63f4af2e659 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:55.927911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 696/1401] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:55.930987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713f7ad-03a2-4019-b4a6-a9c4d1392a26 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:55.931436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713f7ad-03a2-4019-b4a6-a9c4d1392a26 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:55.931613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713f7ad-03a2-4019-b4a6-a9c4d1392a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:56.013017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c713f7ad-03a2-4019-b4a6-a9c4d1392a26 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:56.014390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c713f7ad-03a2-4019-b4a6-a9c4d1392a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:56.132434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 706/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:08:56.430710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd43cf-65ef-490b-896b-c63f4af2e659 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:56.432308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-05cd43cf-65ef-490b-896b-c63f4af2e659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:56.608346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd43cf-65ef-490b-896b-c63f4af2e659 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:56.691666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:56.739294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:08:56.740004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:56.740290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:56.740560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:56.742622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:56.847797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:08:56.849403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e9457-e1b9-4394-be03-accc016df97c service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:08:56.881077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 697/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:08:59.226703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7def17d-168c-41ea-98b5-f5ba09701e5b None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:59.227656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7def17d-168c-41ea-98b5-f5ba09701e5b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:59.227656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7def17d-168c-41ea-98b5-f5ba09701e5b None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:59.228769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 707/1404] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:08:59.234068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f896-b002-4fd9-a688-be735b1ed2e0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:08:59.234552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f896-b002-4fd9-a688-be735b1ed2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:08:59.234647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f896-b002-4fd9-a688-be735b1ed2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:08:59.304441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5a23f896-b002-4fd9-a688-be735b1ed2e0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:08:59.305662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5a23f896-b002-4fd9-a688-be735b1ed2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:59.348873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:59.414342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 698/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:08:59.706939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7def17d-168c-41ea-98b5-f5ba09701e5b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:08:59.709983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f7def17d-168c-41ea-98b5-f5ba09701e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:08:59.755852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:08:59.876529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7def17d-168c-41ea-98b5-f5ba09701e5b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:08:59.998967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:00.001833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:00.003199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:00.008841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:00.008841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:00.008841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:00.108140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:00.109809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886b179-354c-479d-abdf-7df2835c4b4f service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:00.129973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 708/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:02.621944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024222b-92d9-49fd-bb7e-403e8c6b47cb None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:02.622509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024222b-92d9-49fd-bb7e-403e8c6b47cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:02.622509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024222b-92d9-49fd-bb7e-403e8c6b47cb None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:02.623885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 699/1407] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:02.634327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0c0ea-aa04-47c8-95c4-6f4319ddcdee None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:02.635250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0c0ea-aa04-47c8-95c4-6f4319ddcdee None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:02.635751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0c0ea-aa04-47c8-95c4-6f4319ddcdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:02.708375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7eb0c0ea-aa04-47c8-95c4-6f4319ddcdee None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:02.709533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7eb0c0ea-aa04-47c8-95c4-6f4319ddcdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:02.785054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:02.820380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 709/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:03.116044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024222b-92d9-49fd-bb7e-403e8c6b47cb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:03.117588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7024222b-92d9-49fd-bb7e-403e8c6b47cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:03.286920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024222b-92d9-49fd-bb7e-403e8c6b47cb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:03.309917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:03.424850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:03.425810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:03.426309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:03.426584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:03.428317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:03.534470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:03.537314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ea157-c3c8-43ef-b0ea-09fa68d72b76 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:03.566772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 700/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:12.836485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=11 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:13.564403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:16.356273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633e01-b213-44a0-8dce-ed28d43d31ca None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:16.356845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633e01-b213-44a0-8dce-ed28d43d31ca None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:16.356845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633e01-b213-44a0-8dce-ed28d43d31ca None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:16.358203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 710/1410] 10.222.0.62 () {54 vars in 918 bytes} [Tue Sep 13 10:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:16.363506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e68a-c1ea-4b0e-aad6-d51314d23bde None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:16.363951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e68a-c1ea-4b0e-aad6-d51314d23bde None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:16.364055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e68a-c1ea-4b0e-aad6-d51314d23bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:16.441091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-01c9e68a-c1ea-4b0e-aad6-d51314d23bde None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:16.442298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-01c9e68a-c1ea-4b0e-aad6-d51314d23bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:16.546096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 701/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Tue Sep 13 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:16.560858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b76f00-55a2-4ea2-a924-e01b0fe1fc81 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:16.561142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b76f00-55a2-4ea2-a924-e01b0fe1fc81 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:16.561418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b76f00-55a2-4ea2-a924-e01b0fe1fc81 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:16.562741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 711/1412] 10.222.0.62 () {54 vars in 918 bytes} [Tue Sep 13 10:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:16.567567 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8a137-5f47-4059-a865-f70b11d913a3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:16.567851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8a137-5f47-4059-a865-f70b11d913a3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:16.568118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8a137-5f47-4059-a865-f70b11d913a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:16.644107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:16.646515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-19a8a137-5f47-4059-a865-f70b11d913a3 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:16.647770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-19a8a137-5f47-4059-a865-f70b11d913a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:16.765175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 702/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Tue Sep 13 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:09:19.337851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e194-3db4-4511-9dae-56892671675c None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:19.338414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e194-3db4-4511-9dae-56892671675c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:19.338414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e194-3db4-4511-9dae-56892671675c None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:19.339774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:19.345218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93a5b-83f0-4e88-a9d8-531562cfa87d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:19.346107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93a5b-83f0-4e88-a9d8-531562cfa87d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:19.346305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93a5b-83f0-4e88-a9d8-531562cfa87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:19.412698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-78e93a5b-83f0-4e88-a9d8-531562cfa87d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:19.413920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-78e93a5b-83f0-4e88-a9d8-531562cfa87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:19.467609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:19.510018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:19.795931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e194-3db4-4511-9dae-56892671675c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:19.797483 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d648e194-3db4-4511-9dae-56892671675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:19.942299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e194-3db4-4511-9dae-56892671675c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:20.014353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:20.055117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:20.055690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:20.055970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:20.056242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:20.057847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:20.147296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:20.148897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70d633-556b-4c37-a65f-3aff67149b67 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:20.167562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 713/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:21.309618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5120cf-e22d-4140-83ca-4c5003357bca None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:21.310200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5120cf-e22d-4140-83ca-4c5003357bca None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:21.310200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5120cf-e22d-4140-83ca-4c5003357bca None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:21.311554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:21.318134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1f6b-0a61-48f6-b7e2-ee6ac77065d0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:21.318555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1f6b-0a61-48f6-b7e2-ee6ac77065d0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:21.318722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1f6b-0a61-48f6-b7e2-ee6ac77065d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:21.398064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-af1c1f6b-0a61-48f6-b7e2-ee6ac77065d0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:21.399319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-af1c1f6b-0a61-48f6-b7e2-ee6ac77065d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:21.427127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:21.496557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:21.649160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5120cf-e22d-4140-83ca-4c5003357bca None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:21.650596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ec5120cf-e22d-4140-83ca-4c5003357bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:21.795080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5120cf-e22d-4140-83ca-4c5003357bca None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:21.835079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:21.907074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:21.907681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:21.907960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:21.908228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:21.910086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:21.997185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:21.998761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf0bd7-7efd-4e36-a568-4848439d793c service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:22.017538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 705/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:23.143366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72ea5b-a741-4734-b46a-a9e924c9dd77 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:23.143889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72ea5b-a741-4734-b46a-a9e924c9dd77 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:23.143889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72ea5b-a741-4734-b46a-a9e924c9dd77 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:23.145267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:23.151594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764af625-3131-4b25-896b-c51f8460fbe1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:23.156474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764af625-3131-4b25-896b-c51f8460fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:23.156474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764af625-3131-4b25-896b-c51f8460fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:23.207272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:23.229502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-764af625-3131-4b25-896b-c51f8460fbe1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:23.230716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-764af625-3131-4b25-896b-c51f8460fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:23.328168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:23.480113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72ea5b-a741-4734-b46a-a9e924c9dd77 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:23.481580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0e72ea5b-a741-4734-b46a-a9e924c9dd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:23.627094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72ea5b-a741-4734-b46a-a9e924c9dd77 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:23.639259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:23.741066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:23.741669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:23.741947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:23.742219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:23.743848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:23.831955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:23.833589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090e9a7-a93c-4c91-93a0-a643c606a022 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:23.851232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 716/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:24.963240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09fc0c-7098-4412-8099-1d508d3741b4 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:24.963803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09fc0c-7098-4412-8099-1d508d3741b4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:24.963803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09fc0c-7098-4412-8099-1d508d3741b4 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:24.965150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:24.971424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607a54-47bd-4bc1-9177-681f9c206355 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:24.971845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607a54-47bd-4bc1-9177-681f9c206355 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:24.971978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607a54-47bd-4bc1-9177-681f9c206355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:24.991533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:25.053749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e3607a54-47bd-4bc1-9177-681f9c206355 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:25.054994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e3607a54-47bd-4bc1-9177-681f9c206355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:25.150670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:25.301263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09fc0c-7098-4412-8099-1d508d3741b4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:25.302821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dd09fc0c-7098-4412-8099-1d508d3741b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:25.421648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:25.458095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09fc0c-7098-4412-8099-1d508d3741b4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:25.570070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:25.570597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:25.570881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:25.571153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:25.572816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:25.647934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:25.661560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:25.665609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6b002-a307-4237-b3f3-cd1f82e292f4 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:25.681628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 708/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:26.804248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7455-b100-4713-ab8b-18a284ee1b17 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:26.804862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7455-b100-4713-ab8b-18a284ee1b17 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:26.804862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7455-b100-4713-ab8b-18a284ee1b17 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:26.806163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:26.812975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41354a0-a47b-4d96-8d42-4d008b7eefc4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:26.813447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41354a0-a47b-4d96-8d42-4d008b7eefc4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:26.813594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41354a0-a47b-4d96-8d42-4d008b7eefc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:26.889110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f41354a0-a47b-4d96-8d42-4d008b7eefc4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:26.890329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f41354a0-a47b-4d96-8d42-4d008b7eefc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:26.985125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:27.137487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c7455-b100-4713-ab8b-18a284ee1b17 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:27.139059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d79c7455-b100-4713-ab8b-18a284ee1b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:27.230134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:27.293458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c7455-b100-4713-ab8b-18a284ee1b17 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:27.408127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:27.408979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:27.409253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:27.409520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:27.411112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:27.459627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:27.500399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:27.502556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fe271-1c6b-49a6-b862-60c5d654d994 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:27.520323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 719/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:28.638028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c6e2-c0f2-40b5-b55f-576fba2cbbc3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:28.638604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c6e2-c0f2-40b5-b55f-576fba2cbbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:28.638604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c6e2-c0f2-40b5-b55f-576fba2cbbc3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:28.639952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:28.646215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1578a-260c-4bd7-bb2c-3e8b97d825dd None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:28.646640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1578a-260c-4bd7-bb2c-3e8b97d825dd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:28.646767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1578a-260c-4bd7-bb2c-3e8b97d825dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:28.721856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a9b1578a-260c-4bd7-bb2c-3e8b97d825dd None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:28.723082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a9b1578a-260c-4bd7-bb2c-3e8b97d825dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:28.818602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:28.980551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c6e2-c0f2-40b5-b55f-576fba2cbbc3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:28.982034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6f20c6e2-c0f2-40b5-b55f-576fba2cbbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:29.030279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:29.125379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c6e2-c0f2-40b5-b55f-576fba2cbbc3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:29.235280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:29.235896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:29.236171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:29.236437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:29.238023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:29.245241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:29.332939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:29.334527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2d40b-c104-421c-a302-28c4757f509c service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:29.351973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 711/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:30.453166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a61d7-a80a-424f-a376-483dcaa43a06 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:30.453733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a61d7-a80a-424f-a376-483dcaa43a06 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:30.453733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a61d7-a80a-424f-a376-483dcaa43a06 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:30.455191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:30.461330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62ed09-c5a3-49ca-a04f-a579c0e793ec None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:30.461748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62ed09-c5a3-49ca-a04f-a579c0e793ec None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:30.461888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62ed09-c5a3-49ca-a04f-a579c0e793ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:30.537717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4c62ed09-c5a3-49ca-a04f-a579c0e793ec None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:30.538930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4c62ed09-c5a3-49ca-a04f-a579c0e793ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:30.606184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:30.635587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:30.785829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a61d7-a80a-424f-a376-483dcaa43a06 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:30.787452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-278a61d7-a80a-424f-a376-483dcaa43a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:30.805665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:30.934208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a61d7-a80a-424f-a376-483dcaa43a06 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:31.020497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:31.046573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:31.047203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:31.047492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:31.048371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:31.049979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:31.138068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:31.139698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcefdd-65d5-4420-b661-1033e9c6f5e2 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:31.159060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 722/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:32.259420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7938959-f043-4cf4-a303-48d22c49a52d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:32.259995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7938959-f043-4cf4-a303-48d22c49a52d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:32.259995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7938959-f043-4cf4-a303-48d22c49a52d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:32.261322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:32.268333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392ca9c-4799-46c4-a342-56cc76ab345f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:32.268791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392ca9c-4799-46c4-a342-56cc76ab345f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:32.268913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392ca9c-4799-46c4-a342-56cc76ab345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:32.343981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e392ca9c-4799-46c4-a342-56cc76ab345f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:32.345284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e392ca9c-4799-46c4-a342-56cc76ab345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:32.389004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:32.443826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:32.596773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7938959-f043-4cf4-a303-48d22c49a52d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:32.598207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c7938959-f043-4cf4-a303-48d22c49a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:32.742456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7938959-f043-4cf4-a303-48d22c49a52d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:32.796763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:32.854662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:32.855226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:32.855514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:32.855782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:32.857383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:32.946020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:32.947631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5294ca-b203-480b-b9c0-72379db4ed08 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:32.965132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 714/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:34.047155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c92a-7094-4f5f-9df7-5be8c4c6887a None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:34.047722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c92a-7094-4f5f-9df7-5be8c4c6887a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:34.047722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c92a-7094-4f5f-9df7-5be8c4c6887a None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:34.049101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:34.055313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dbdc3-d385-489e-b098-166336961070 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:34.055780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dbdc3-d385-489e-b098-166336961070 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:34.055888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dbdc3-d385-489e-b098-166336961070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:34.132477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6b1dbdc3-d385-489e-b098-166336961070 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:34.133697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6b1dbdc3-d385-489e-b098-166336961070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:34.135711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:34.228513 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:34.380075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c92a-7094-4f5f-9df7-5be8c4c6887a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:34.381804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bd64c92a-7094-4f5f-9df7-5be8c4c6887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:34.529518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c92a-7094-4f5f-9df7-5be8c4c6887a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:34.560500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:34.646023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:34.646561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:34.646841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:34.647111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:34.648863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:34.738027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:34.739749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d23aa-013f-41d7-81b0-1baeb6dc5bca service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:34.757609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 725/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:36.824523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3dda3-65c4-4b67-a73b-b282970b8c20 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:36.825124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3dda3-65c4-4b67-a73b-b282970b8c20 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:36.825124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3dda3-65c4-4b67-a73b-b282970b8c20 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:36.826430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:36.832045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a897098-f808-4e40-83d8-b6a42203b304 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:36.832463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a897098-f808-4e40-83d8-b6a42203b304 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:36.832576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a897098-f808-4e40-83d8-b6a42203b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:36.873691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:36.899689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7a897098-f808-4e40-83d8-b6a42203b304 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:36.900931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7a897098-f808-4e40-83d8-b6a42203b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:36.997733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:09:37.288040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3dda3-65c4-4b67-a73b-b282970b8c20 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:37.289583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3fd3dda3-65c4-4b67-a73b-b282970b8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:37.392270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3dda3-65c4-4b67-a73b-b282970b8c20 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:37.426104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:37.466107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:37.466712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:37.466989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:37.467284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:37.468874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:37.576604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:37.577989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4630c-c5e2-4999-b0b2-4f3e91bfe221 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:37.596538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 717/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:39.970792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299e14-f1b1-451d-8685-41c756a134cb None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:39.971384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299e14-f1b1-451d-8685-41c756a134cb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:39.971384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299e14-f1b1-451d-8685-41c756a134cb None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:39.972759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:39.977927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd5f2a-370e-401c-b499-315b98f7b643 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:39.978364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd5f2a-370e-401c-b499-315b98f7b643 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:39.978475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd5f2a-370e-401c-b499-315b98f7b643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:40.044639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-21fd5f2a-370e-401c-b499-315b98f7b643 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:40.045855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-21fd5f2a-370e-401c-b499-315b98f7b643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:40.058943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:40.146167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:40.435457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299e14-f1b1-451d-8685-41c756a134cb None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:40.437043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6b299e14-f1b1-451d-8685-41c756a134cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:40.540977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299e14-f1b1-451d-8685-41c756a134cb None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:40.553792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:40.614261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:40.614867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:40.615148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:40.615450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:40.617062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:40.725457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:40.726810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4c19c-395a-4f8b-aba8-fa7e0c7657fc service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:40.744433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 728/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:43.267417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec5930-0ac3-4e4c-8409-8180b3cc60ce None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:43.267995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec5930-0ac3-4e4c-8409-8180b3cc60ce None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:43.267995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec5930-0ac3-4e4c-8409-8180b3cc60ce None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:43.269329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:43.274829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489537d4-6a8f-452d-84ad-cd9016d3bdf4 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:43.275271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489537d4-6a8f-452d-84ad-cd9016d3bdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:43.275450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489537d4-6a8f-452d-84ad-cd9016d3bdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:43.325064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:43.343403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-489537d4-6a8f-452d-84ad-cd9016d3bdf4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:43.345450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-489537d4-6a8f-452d-84ad-cd9016d3bdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:43.443188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:43.725304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec5930-0ac3-4e4c-8409-8180b3cc60ce None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:43.726876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0fec5930-0ac3-4e4c-8409-8180b3cc60ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:43.828022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec5930-0ac3-4e4c-8409-8180b3cc60ce None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:43.868096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:43.900367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:43.901243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:43.901707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:43.902329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:43.903969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:44.010156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:44.011608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1820a1d-8620-4aea-a658-894dc27f71f5 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:44.029988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 720/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:46.308388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c818ad-443f-4d8c-aae6-d679f353ede7 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:46.308958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c818ad-443f-4d8c-aae6-d679f353ede7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:46.308958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c818ad-443f-4d8c-aae6-d679f353ede7 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:46.310298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:46.316283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e37bd-de15-4b38-90e7-47c58770dc2f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:46.316729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e37bd-de15-4b38-90e7-47c58770dc2f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:46.316847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e37bd-de15-4b38-90e7-47c58770dc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:46.382152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-599e37bd-de15-4b38-90e7-47c58770dc2f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:46.383386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-599e37bd-de15-4b38-90e7-47c58770dc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:46.404010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:46.478577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:46.764831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c818ad-443f-4d8c-aae6-d679f353ede7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:46.766439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a7c818ad-443f-4d8c-aae6-d679f353ede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:46.871407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c818ad-443f-4d8c-aae6-d679f353ede7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:46.894396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:46.948669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:46.949209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:46.949488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:46.949759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:46.951379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:47.060104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:47.061532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933bc93-514b-45c6-9f66-ad750f483e85 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:47.084520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 731/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:49.365135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a8d8-2580-4d46-b2c4-b4d5a72c55ba None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:49.365714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a8d8-2580-4d46-b2c4-b4d5a72c55ba None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:49.365714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a8d8-2580-4d46-b2c4-b4d5a72c55ba None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:49.367084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:49.372312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a853267-81c4-4824-b53f-6bc8b0d3ab26 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:49.372758 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a853267-81c4-4824-b53f-6bc8b0d3ab26 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:49.372867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a853267-81c4-4824-b53f-6bc8b0d3ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:49.436762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:49.440002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5a853267-81c4-4824-b53f-6bc8b0d3ab26 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:49.441231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5a853267-81c4-4824-b53f-6bc8b0d3ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:49.539480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:49.831997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a8d8-2580-4d46-b2c4-b4d5a72c55ba None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:49.833621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d7b3a8d8-2580-4d46-b2c4-b4d5a72c55ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:49.934956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a8d8-2580-4d46-b2c4-b4d5a72c55ba None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:49.983333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:50.007733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:50.008396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:50.008677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:50.008949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:50.010591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:50.118528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:50.119911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a5891-8c6d-48a8-a306-54743f1c2605 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:50.138660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 723/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:09:52.352424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d24299-a85f-48f5-86f3-d09ac316f6fd None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:52.352964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d24299-a85f-48f5-86f3-d09ac316f6fd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:52.352964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d24299-a85f-48f5-86f3-d09ac316f6fd None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:52.354307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:52.360231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389f456-ee55-4e75-9542-bdd2f981e338 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:52.360636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389f456-ee55-4e75-9542-bdd2f981e338 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:52.360812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389f456-ee55-4e75-9542-bdd2f981e338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:52.427856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b389f456-ee55-4e75-9542-bdd2f981e338 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:52.429085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b389f456-ee55-4e75-9542-bdd2f981e338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:52.458219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:52.525092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:52.806976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d24299-a85f-48f5-86f3-d09ac316f6fd None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:52.808606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-44d24299-a85f-48f5-86f3-d09ac316f6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:52.912740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d24299-a85f-48f5-86f3-d09ac316f6fd None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:52.940830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:52.986144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:52.986749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:52.987026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:52.987330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:52.988930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:53.097914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:53.099309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e0ad5-a1b4-448b-b832-c9f127fcdd92 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:53.117393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 734/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:55.331714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1147-2922-4b95-9c77-9498325f422d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:55.332287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1147-2922-4b95-9c77-9498325f422d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:55.332287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1147-2922-4b95-9c77-9498325f422d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:55.333626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:55.338860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c3f86-9cea-4980-99b6-76c827bef6ba None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:55.339267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c3f86-9cea-4980-99b6-76c827bef6ba None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:55.339463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c3f86-9cea-4980-99b6-76c827bef6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:55.406128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-248c3f86-9cea-4980-99b6-76c827bef6ba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:55.407390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-248c3f86-9cea-4980-99b6-76c827bef6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:55.415940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:55.514908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:55.809135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a1147-2922-4b95-9c77-9498325f422d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:55.810755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-db8a1147-2922-4b95-9c77-9498325f422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:55.921903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a1147-2922-4b95-9c77-9498325f422d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:55.978557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:55.996487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:55.997122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:55.997401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:55.997694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:56.000154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:56.106964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:56.108428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f6a0-1d7d-4537-aa99-da2999440d6a service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:56.126671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 726/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:09:58.352908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3d6c-0f7c-4566-9f4e-8c894243fd0d None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:58.353582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3d6c-0f7c-4566-9f4e-8c894243fd0d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:58.353582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3d6c-0f7c-4566-9f4e-8c894243fd0d None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:58.354806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:09:58.360648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3372d-f232-47b3-a23e-62c9fcd93c8d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:58.361048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3372d-f232-47b3-a23e-62c9fcd93c8d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:58.361200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3372d-f232-47b3-a23e-62c9fcd93c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:58.427334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-57e3372d-f232-47b3-a23e-62c9fcd93c8d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:09:58.428532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-57e3372d-f232-47b3-a23e-62c9fcd93c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:58.466673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:58.523799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:09:58.807702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3d6c-0f7c-4566-9f4e-8c894243fd0d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:09:58.809254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-30ff3d6c-0f7c-4566-9f4e-8c894243fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:58.911807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3d6c-0f7c-4566-9f4e-8c894243fd0d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:09:58.949144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:09:58.985083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:09:58.985694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:09:58.985978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:09:58.986259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:09:58.988585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:09:59.095973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:09:59.097341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7f04f-e547-4b94-8d7a-8bf79b8a27c0 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:09:59.117099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 737/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:01.295888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3913a-1d0b-4678-a528-dc47ec6e91d3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:01.296457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3913a-1d0b-4678-a528-dc47ec6e91d3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:01.296575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3913a-1d0b-4678-a528-dc47ec6e91d3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:01.297850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:01.303500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00698a3b-55ff-492d-b53b-b6c49699427e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:01.303896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00698a3b-55ff-492d-b53b-b6c49699427e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:01.304057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00698a3b-55ff-492d-b53b-b6c49699427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:01.369135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-00698a3b-55ff-492d-b53b-b6c49699427e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:01.370351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-00698a3b-55ff-492d-b53b-b6c49699427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:01.388461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:01.466684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:01.757294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3913a-1d0b-4678-a528-dc47ec6e91d3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:01.758845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-61a3913a-1d0b-4678-a528-dc47ec6e91d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:01.862834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3913a-1d0b-4678-a528-dc47ec6e91d3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:01.928360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:01.934684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:01.935338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:01.935620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:01.935894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:01.937497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:02.044841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:02.046218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801acfc-8b2c-4883-9280-5e6a17c2b4fa service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:02.063644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 729/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:04.229356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90471b-c88f-4393-8ab2-da4784d98130 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:04.229900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90471b-c88f-4393-8ab2-da4784d98130 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:04.229900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90471b-c88f-4393-8ab2-da4784d98130 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:04.231294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:04.237080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff7d6-7cda-4f85-b047-eb009c04555e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:04.237523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff7d6-7cda-4f85-b047-eb009c04555e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:04.237641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff7d6-7cda-4f85-b047-eb009c04555e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:04.303876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c9fff7d6-7cda-4f85-b047-eb009c04555e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:04.305089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c9fff7d6-7cda-4f85-b047-eb009c04555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:04.353774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:04.401740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:04.686523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90471b-c88f-4393-8ab2-da4784d98130 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:04.688162 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cd90471b-c88f-4393-8ab2-da4784d98130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:04.790720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90471b-c88f-4393-8ab2-da4784d98130 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:04.837781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:04.864641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:04.865249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:04.865532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:04.865808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:04.867446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:04.979450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:04.980823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4025e-e2fe-481a-94ae-f8bb6c6c32b3 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:04.999511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 740/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:14.400201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=22 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:14.997141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=66 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:24.918821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66803828-d7e9-426d-a4ca-a7f138661d12 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:24.919520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66803828-d7e9-426d-a4ca-a7f138661d12 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:24.919520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66803828-d7e9-426d-a4ca-a7f138661d12 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:24.921406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 731/1471] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:24.926574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930a87-061a-4766-8964-c7e1cd95c0b5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:24.927013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930a87-061a-4766-8964-c7e1cd95c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:24.927108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930a87-061a-4766-8964-c7e1cd95c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:24.992303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-33930a87-061a-4766-8964-c7e1cd95c0b5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:24.993505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-33930a87-061a-4766-8964-c7e1cd95c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:25.089099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 741/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:25.388210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc8f4-b499-4175-ae2a-27b6c94067d8 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:25.388571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc8f4-b499-4175-ae2a-27b6c94067d8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:25.389408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc8f4-b499-4175-ae2a-27b6c94067d8 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:25.390833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 732/1473] 10.222.0.62 () {54 vars in 966 bytes} [Tue Sep 13 10:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:25.395894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39bb4a-1574-434f-b5dd-3dd6583f350d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:25.396176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39bb4a-1574-434f-b5dd-3dd6583f350d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:25.396446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39bb4a-1574-434f-b5dd-3dd6583f350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:25.471719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5f39bb4a-1574-434f-b5dd-3dd6583f350d None None] MFA Rules not processed for user `5950076b981b43feba3185f1fb513729`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:25.472932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5f39bb4a-1574-434f-b5dd-3dd6583f350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:25.493110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:25.550273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 742/1474] 10.222.0.62 () {58 vars in 1063 bytes} [Tue Sep 13 10:10:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:25.558143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc8f4-b499-4175-ae2a-27b6c94067d8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:25.560003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9a7bc8f4-b499-4175-ae2a-27b6c94067d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:25.664465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc8f4-b499-4175-ae2a-27b6c94067d8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:25.738782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:25.739431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:25.739711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:25.739982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:25.741619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:25.797474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:25.854170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:25.855679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f81e36-474f-4345-a3cb-eb53a51680e3 service glance] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:25.876508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 733/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:28.159506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c50b-b44c-4689-8633-c8a70b6c4dfa None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:28.160094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c50b-b44c-4689-8633-c8a70b6c4dfa None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:28.160094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c50b-b44c-4689-8633-c8a70b6c4dfa None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:28.161421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 743/1476] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:28.166825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df9ab8-d0ce-439c-9d0f-e1b22824dc37 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:28.167291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df9ab8-d0ce-439c-9d0f-e1b22824dc37 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:28.167396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df9ab8-d0ce-439c-9d0f-e1b22824dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:28.234367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-07df9ab8-d0ce-439c-9d0f-e1b22824dc37 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:28.235634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-07df9ab8-d0ce-439c-9d0f-e1b22824dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:28.332107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 734/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:28.610624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c50b-b44c-4689-8633-c8a70b6c4dfa None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:28.616309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2e07c50b-b44c-4689-8633-c8a70b6c4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:28.738581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c50b-b44c-4689-8633-c8a70b6c4dfa None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:28.789196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:28.832507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:28.833113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:28.833391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:28.833661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:28.834291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:28.836114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:10:28.836114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:10:28.836114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:10:28.837257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cadec-53bf-471c-9399-f681f8dd9292 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:28.856270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 744/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Tue Sep 13 10:10:28 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:10:31.034129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264e920-4e94-4496-aa11-a0d41b727b9f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:31.034701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264e920-4e94-4496-aa11-a0d41b727b9f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:31.034701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264e920-4e94-4496-aa11-a0d41b727b9f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:31.036083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 735/1479] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:31.041343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a7d35-c839-49b9-9525-f249109b12df None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:31.041787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a7d35-c839-49b9-9525-f249109b12df None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:31.041899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a7d35-c839-49b9-9525-f249109b12df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:31.112730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5d4a7d35-c839-49b9-9525-f249109b12df None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:31.113972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5d4a7d35-c839-49b9-9525-f249109b12df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:31.210382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 745/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:31.502842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264e920-4e94-4496-aa11-a0d41b727b9f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:31.504411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0264e920-4e94-4496-aa11-a0d41b727b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:31.514054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:31.650575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264e920-4e94-4496-aa11-a0d41b727b9f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:31.731069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:31.764412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99757739-d98c-4107-951f-39311ae643ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:31.765016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99757739-d98c-4107-951f-39311ae643ff service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:31.765296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99757739-d98c-4107-951f-39311ae643ff service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:31.765566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99757739-d98c-4107-951f-39311ae643ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:31.767202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-99757739-d98c-4107-951f-39311ae643ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:31.855579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99757739-d98c-4107-951f-39311ae643ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:31.857225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99757739-d98c-4107-951f-39311ae643ff service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:31.875510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 736/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:34.217545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd490-6fbb-4b8a-84ab-a47be5c2255b None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:34.218109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd490-6fbb-4b8a-84ab-a47be5c2255b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:34.218109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd490-6fbb-4b8a-84ab-a47be5c2255b None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:34.219496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 746/1482] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:34.224906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74f23c-2670-4c0d-b2c9-6e4c18e3c000 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:34.225339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74f23c-2670-4c0d-b2c9-6e4c18e3c000 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:34.225438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74f23c-2670-4c0d-b2c9-6e4c18e3c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:34.291588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8b74f23c-2670-4c0d-b2c9-6e4c18e3c000 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:34.292813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8b74f23c-2670-4c0d-b2c9-6e4c18e3c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:34.339988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:34.390678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 737/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:34.676265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd490-6fbb-4b8a-84ab-a47be5c2255b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:34.677839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c9edd490-6fbb-4b8a-84ab-a47be5c2255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:34.701176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:34.779887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd490-6fbb-4b8a-84ab-a47be5c2255b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:34.852393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:34.852997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:34.853276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:34.853549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:34.855128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:34.929559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:34.963733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:34.965170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7276ba-121c-45be-a3f6-191f2c2f2990 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:35.102332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 747/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:37.302225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d8f2-a91f-4a58-9244-46b8e56a8009 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:37.302785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d8f2-a91f-4a58-9244-46b8e56a8009 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:37.302785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d8f2-a91f-4a58-9244-46b8e56a8009 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:37.304164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 738/1485] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:37.309652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2587a-d10f-435e-a2d6-593f664cbfe6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:37.310073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2587a-d10f-435e-a2d6-593f664cbfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:37.310203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2587a-d10f-435e-a2d6-593f664cbfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:37.376027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4ae2587a-d10f-435e-a2d6-593f664cbfe6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:37.377243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4ae2587a-d10f-435e-a2d6-593f664cbfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:37.473374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 748/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:37.767375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d8f2-a91f-4a58-9244-46b8e56a8009 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:37.768942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4cc2d8f2-a91f-4a58-9244-46b8e56a8009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:37.872450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d8f2-a91f-4a58-9244-46b8e56a8009 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:37.943917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:37.945397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:37.946003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:37.946283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:37.946554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:37.948189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:38.056270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:38.057633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dd0df-2545-4374-95c3-d0abc06cf278 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:38.075209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 739/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:40.321878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7287033-c520-4fe3-82c5-2703091251aa None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:40.322450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7287033-c520-4fe3-82c5-2703091251aa None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:40.322450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7287033-c520-4fe3-82c5-2703091251aa None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:40.323860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 749/1488] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:40.329697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e60fa-18d9-4352-8930-5d3199ba85ef None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:40.330133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e60fa-18d9-4352-8930-5d3199ba85ef None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:40.330231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e60fa-18d9-4352-8930-5d3199ba85ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:40.396257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-771e60fa-18d9-4352-8930-5d3199ba85ef None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:40.397494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-771e60fa-18d9-4352-8930-5d3199ba85ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:40.449856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:40.494706 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 740/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:10:40.788249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7287033-c520-4fe3-82c5-2703091251aa None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:40.789823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d7287033-c520-4fe3-82c5-2703091251aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:40.825916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:40.894792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7287033-c520-4fe3-82c5-2703091251aa None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:40.967934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:40.968538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:40.968818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:40.969091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:40.970692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:41.050727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:41.079629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:41.081059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9375c-7be5-48b3-9e27-b644bd693b3c service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:41.102694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 750/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:43.581305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dff139-db06-4a2d-80ba-51481e92d8df None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:43.581888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dff139-db06-4a2d-80ba-51481e92d8df None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:43.581888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dff139-db06-4a2d-80ba-51481e92d8df None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:43.583211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 741/1491] 10.222.0.62 () {54 vars in 927 bytes} [Tue Sep 13 10:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:10:43.588487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e0a40-9925-4932-be96-4c9e9705691c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:43.588880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e0a40-9925-4932-be96-4c9e9705691c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:43.589041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e0a40-9925-4932-be96-4c9e9705691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:43.658051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-147e0a40-9925-4932-be96-4c9e9705691c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:10:43.659315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-147e0a40-9925-4932-be96-4c9e9705691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:43.757298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 751/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Tue Sep 13 10:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:10:44.048597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dff139-db06-4a2d-80ba-51481e92d8df None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:10:44.050493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c1dff139-db06-4a2d-80ba-51481e92d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:44.152538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dff139-db06-4a2d-80ba-51481e92d8df None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:10:44.223925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:10:44.224524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:10:44.224799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:10:44.225065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:10:44.226645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:10:44.236767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:44.335185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:10:44.336540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac478ed3-b9f4-400e-817b-86f89d404f4d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:10:44.354697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 742/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:10:53.755775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=82 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:10:54.352403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=50 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:06.394268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:06.394754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:06.394864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:06.468813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:12:06.470020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:06.562661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 752/1494] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:12:06.570710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:06.571271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:06.571392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:06.647626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:12:06.648777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:06.747611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 743/1495] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:12:06.752235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:06.753675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:06.765456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672d1b7-6230-4915-8614-b9dd12814633 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:06.827584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:06.860732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:06.861332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:06.861611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:06.861888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:06.862558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:12:06.862862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:06.863916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:06.863916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:12:06.863916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:12:06.864803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:06.876390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 753/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:12:06.884809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:06.894627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8c39-bd1b-4069-9b0f-35e2dfa9a66d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:06.957521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:06.990057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:06.990662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:06.990937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:06.991228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:06.991857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:06.993543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:06.993543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:12:06.993543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:12:06.994729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:07.115680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 744/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:12:06 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:12:07.120067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:07.122189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:07.155347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f72472-d104-44df-9e38-2ccca7ae8748 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:07.250050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:07.250674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:07.250951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:07.251256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:07.251934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:07.252950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:07.252950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:12:07.252950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:12:07.254164 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:07.373352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 754/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:12:07 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:12:07.377614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:07.379128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:07.440873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01c551-9d04-45cc-a1e7-b61767048b00 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:07.523186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:07.533975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:07.534579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:07.534860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:07.535133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:07.535805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:07.536805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:07.536805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:12:07.536805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:12:07.537743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:07.552130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 745/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:12:07.555897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:07.557429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:07.568920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948060a-36a6-4bcf-8fa2-4874478722f9 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:07.594950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:07.662970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:07.663610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:07.663889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:07.664160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:07.664803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:07.665812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:07.665812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:12:07.665812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:12:07.666734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27f49b-a5ba-4404-ae88-3c7b8c5e4b69 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:07.679946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 755/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:12:07.683945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:07.685476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:07.718146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13088bd2-82c1-4a01-86cf-e886c911e4c2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:07.811668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:07.812251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:07.812532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:07.812805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] PATH_INFO: `/v3/projects/bbd873877b474a30987b09137f73592a/users/554479a507b24312aa35179b0627daeb/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:07.835270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd873877b474a30987b09137f73592a, user_id=554479a507b24312aa35179b0627daeb, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:07.836284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:07.836284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:12:07.836284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:12:07.838226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8389e-567f-47cd-a8c0-24b30c1ba590 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:07.980649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 746/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:12:07 2022] PUT /identity/v3/projects/bbd873877b474a30987b09137f73592a/users/554479a507b24312aa35179b0627daeb/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:12:07.993021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d3ea7-06b6-4ed3-9cc2-dadb639e66b5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:07.993314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d3ea7-06b6-4ed3-9cc2-dadb639e66b5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:07.993588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d3ea7-06b6-4ed3-9cc2-dadb639e66b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:08.050372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-938d3ea7-06b6-4ed3-9cc2-dadb639e66b5 None None] MFA Rules not processed for user `554479a507b24312aa35179b0627daeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:12:08.051627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-938d3ea7-06b6-4ed3-9cc2-dadb639e66b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:08.127152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 756/1502] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:12:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:12:08.140669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c3192-7549-4614-b844-2c59f75ef60b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:08.140979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c3192-7549-4614-b844-2c59f75ef60b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:08.141257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c3192-7549-4614-b844-2c59f75ef60b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:08.142604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 747/1503] 10.222.0.62 () {54 vars in 964 bytes} [Tue Sep 13 10:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:12:08.147346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ae5c5-ec46-4f00-97b9-5eeb86c7be9e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:08.147635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ae5c5-ec46-4f00-97b9-5eeb86c7be9e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:08.147909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ae5c5-ec46-4f00-97b9-5eeb86c7be9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:08.179096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:08.224509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c90ae5c5-ec46-4f00-97b9-5eeb86c7be9e None None] MFA Rules not processed for user `566f088827ab4710bf54d64415831a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:12:08.225714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c90ae5c5-ec46-4f00-97b9-5eeb86c7be9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:08.299526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 757/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Tue Sep 13 10:12:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:12:08.309635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c3192-7549-4614-b844-2c59f75ef60b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:08.311189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-115c3192-7549-4614-b844-2c59f75ef60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:08.361400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:08.414096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c3192-7549-4614-b844-2c59f75ef60b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:08.495000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:08.495646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:08.495922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:08.496190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:08.497785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:08.594880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:08.596413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:08.600002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:08.613975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 748/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:12:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:12:08.659969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde1324-905f-48ba-964b-068181d4b43a None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:08.660376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde1324-905f-48ba-964b-068181d4b43a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:08.660524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde1324-905f-48ba-964b-068181d4b43a None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:08.661823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 758/1506] 10.222.0.62 () {52 vars in 830 bytes} [Tue Sep 13 10:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:12:08.665248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:08.666446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:08.765965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cccab8-e143-4397-9af7-81dfb53333e6 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:08.862291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:08.862926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:08.863240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:08.863520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] PATH_INFO: `/v3/users/554479a507b24312aa35179b0627daeb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:08.877840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=554479a507b24312aa35179b0627daeb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:08.878855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:08.878855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:12:08.878855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:12:08.880126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:08.889738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:09.096193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554479a507b24312aa35179b0627daeb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:09.176924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554479a507b24312aa35179b0627daeb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:09.177451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554479a507b24312aa35179b0627daeb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:09.177883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554479a507b24312aa35179b0627daeb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:09.192509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554479a507b24312aa35179b0627daeb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:09.221042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 749/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:12:08 2022] DELETE /identity/v3/users/554479a507b24312aa35179b0627daeb => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:12:09.225414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822bed6-7786-437c-9120-0da248596061 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:09.225706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822bed6-7786-437c-9120-0da248596061 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:09.225986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822bed6-7786-437c-9120-0da248596061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:09.283628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e822bed6-7786-437c-9120-0da248596061 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:12:09.284862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e822bed6-7786-437c-9120-0da248596061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:09.359889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:09.371783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 759/1508] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:12:09.384449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:09.385685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:09.485094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c90c59-38c2-4e76-8731-404b5f7da7f7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:09.539549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:09.558039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:09.558653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:09.558934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:09.559246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:09.560850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:09.724401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:09.725787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d66d-0b4e-43f6-8bdc-f65a7dbda1ee service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:09.743776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 750/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:12:10.213335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822bed6-7786-437c-9120-0da248596061 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:12:10.215778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e822bed6-7786-437c-9120-0da248596061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:12:10.226656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822bed6-7786-437c-9120-0da248596061 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:12:10.324176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:12:10.324818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:12:10.325099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:12:10.325375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] PATH_INFO: `/v3/projects/bbd873877b474a30987b09137f73592a` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:12:10.335184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd873877b474a30987b09137f73592a)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:12:10.336225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:12:10.336225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:12:10.336225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:12:10.337428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:12:10.445396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd873877b474a30987b09137f73592a', 'request_id': 'req-1442962a-ad74-4e8d-8a13-33d5d6368a9e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:10.459538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd873877b474a30987b09137f73592a', 'request_id': 'req-1442962a-ad74-4e8d-8a13-33d5d6368a9e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:10.527436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd873877b474a30987b09137f73592a', 'request_id': 'req-1442962a-ad74-4e8d-8a13-33d5d6368a9e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:10.552558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] The token cache is being invalidate because project bbd873877b474a30987b09137f73592a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:12:10.552856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:10.553420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1442962a-ad74-4e8d-8a13-33d5d6368a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:12:10.554772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 760/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:12:10 2022] DELETE /identity/v3/projects/bbd873877b474a30987b09137f73592a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:12:19.741435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=62 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:12:20.549514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:13:37.859511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104286ae-2a71-42e5-8e5d-16ae3c5d801f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:37.860114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104286ae-2a71-42e5-8e5d-16ae3c5d801f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:37.860114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104286ae-2a71-42e5-8e5d-16ae3c5d801f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:37.861425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:13:37.866846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214b933-3fe2-4304-b0e6-34090bd4357a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:37.868981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214b933-3fe2-4304-b0e6-34090bd4357a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:37.868981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214b933-3fe2-4304-b0e6-34090bd4357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:37.961344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6214b933-3fe2-4304-b0e6-34090bd4357a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:37.962597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6214b933-3fe2-4304-b0e6-34090bd4357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:38.070611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:38.226975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104286ae-2a71-42e5-8e5d-16ae3c5d801f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:38.234099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-104286ae-2a71-42e5-8e5d-16ae3c5d801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:38.357314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104286ae-2a71-42e5-8e5d-16ae3c5d801f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:38.362723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541593d6-1127-4438-947f-cea1f01ae755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:38.363351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541593d6-1127-4438-947f-cea1f01ae755 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:38.363641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541593d6-1127-4438-947f-cea1f01ae755 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:38.363896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541593d6-1127-4438-947f-cea1f01ae755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:38.371950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541593d6-1127-4438-947f-cea1f01ae755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:38.373531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541593d6-1127-4438-947f-cea1f01ae755 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:38.382810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 752/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:13:39.471004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476932cc-3e2a-4b88-b8a4-7ba39d00976f None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:39.471542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476932cc-3e2a-4b88-b8a4-7ba39d00976f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:39.471666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476932cc-3e2a-4b88-b8a4-7ba39d00976f None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:39.473004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:13:39.480004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0644-c0b6-4d52-b1f1-74591eaf1b9e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:39.480430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0644-c0b6-4d52-b1f1-74591eaf1b9e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:39.480554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0644-c0b6-4d52-b1f1-74591eaf1b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:39.500289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f94a0644-c0b6-4d52-b1f1-74591eaf1b9e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:39.501508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f94a0644-c0b6-4d52-b1f1-74591eaf1b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:39.509743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:39.662572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476932cc-3e2a-4b88-b8a4-7ba39d00976f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:39.678610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476932cc-3e2a-4b88-b8a4-7ba39d00976f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:39.684767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48832f-9bd1-44fd-8720-cfa55ea59044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:39.685367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48832f-9bd1-44fd-8720-cfa55ea59044 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:39.685679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48832f-9bd1-44fd-8720-cfa55ea59044 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:39.685958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48832f-9bd1-44fd-8720-cfa55ea59044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:39.694376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48832f-9bd1-44fd-8720-cfa55ea59044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:39.696033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48832f-9bd1-44fd-8720-cfa55ea59044 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:39.706078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 763/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:13:40.109947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfd297-c87f-40cb-9da7-0e8a42d40148 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:40.110233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfd297-c87f-40cb-9da7-0e8a42d40148 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:40.110510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfd297-c87f-40cb-9da7-0e8a42d40148 None None] PATH_INFO: `/v3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:40.111835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 754/1517] 10.222.0.50 () {54 vars in 925 bytes} [Tue Sep 13 10:13:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:13:40.117991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6eae8-492a-4349-bcab-db9e82a54f6a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:40.118277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6eae8-492a-4349-bcab-db9e82a54f6a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:40.118550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6eae8-492a-4349-bcab-db9e82a54f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:40.180371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-dde6eae8-492a-4349-bcab-db9e82a54f6a None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:40.181576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dde6eae8-492a-4349-bcab-db9e82a54f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:40.197811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:13:40.252124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 764/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Sep 13 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:44.201027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e79b96-8610-4ea4-94ab-1a77ced56f37 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:44.201523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e79b96-8610-4ea4-94ab-1a77ced56f37 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:44.201618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e79b96-8610-4ea4-94ab-1a77ced56f37 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:44.203181 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 755/1519] 10.222.0.62 () {54 vars in 961 bytes} [Tue Sep 13 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:13:44.208349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdff1f-991d-466b-9617-3a7642850afb None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:44.208840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdff1f-991d-466b-9617-3a7642850afb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:44.209040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdff1f-991d-466b-9617-3a7642850afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:44.231439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-dfbdff1f-991d-466b-9617-3a7642850afb None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:44.232954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dfbdff1f-991d-466b-9617-3a7642850afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:44.243312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 765/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Tue Sep 13 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:44.250159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e79b96-8610-4ea4-94ab-1a77ced56f37 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:44.266107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e79b96-8610-4ea4-94ab-1a77ced56f37 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:44.271332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbff548-c8d3-46fb-892a-50d62a6f4d60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:44.271938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbff548-c8d3-46fb-892a-50d62a6f4d60 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:44.272220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbff548-c8d3-46fb-892a-50d62a6f4d60 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:44.272495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbff548-c8d3-46fb-892a-50d62a6f4d60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:44.284452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbff548-c8d3-46fb-892a-50d62a6f4d60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:44.286045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbff548-c8d3-46fb-892a-50d62a6f4d60 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:44.296040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 756/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:13:51.177143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376c1c7-f4f1-499b-ae0a-812739e3d8ee None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:51.177627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376c1c7-f4f1-499b-ae0a-812739e3d8ee None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:51.177726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376c1c7-f4f1-499b-ae0a-812739e3d8ee None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:51.179547 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:13:51.185837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00592e46-1688-4a50-bb55-105611bc54ba None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:51.186282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00592e46-1688-4a50-bb55-105611bc54ba None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:51.186444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00592e46-1688-4a50-bb55-105611bc54ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:51.205479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-00592e46-1688-4a50-bb55-105611bc54ba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:51.206758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-00592e46-1688-4a50-bb55-105611bc54ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:51.216244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:51.372215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376c1c7-f4f1-499b-ae0a-812739e3d8ee None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:51.387156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376c1c7-f4f1-499b-ae0a-812739e3d8ee None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:51.392736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6acde-d10e-45ee-83ce-8181ff4767ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:51.393368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6acde-d10e-45ee-83ce-8181ff4767ae service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:51.393650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6acde-d10e-45ee-83ce-8181ff4767ae service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:51.393926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6acde-d10e-45ee-83ce-8181ff4767ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:51.403035 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6acde-d10e-45ee-83ce-8181ff4767ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:51.404604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6acde-d10e-45ee-83ce-8181ff4767ae service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:51.414159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 767/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:13:52.478881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad274799-539f-4029-a35f-0370179b80d4 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:52.479440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad274799-539f-4029-a35f-0370179b80d4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:52.479440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad274799-539f-4029-a35f-0370179b80d4 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:52.480878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 10:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:13:52.486913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d845-d8c6-4292-83bc-2c08bc66bc15 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:52.487387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d845-d8c6-4292-83bc-2c08bc66bc15 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:52.487493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d845-d8c6-4292-83bc-2c08bc66bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:52.506568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b107d845-d8c6-4292-83bc-2c08bc66bc15 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:52.507815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b107d845-d8c6-4292-83bc-2c08bc66bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:52.515828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 10:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:52.670199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad274799-539f-4029-a35f-0370179b80d4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:52.682822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad274799-539f-4029-a35f-0370179b80d4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:52.688347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4703bdc-6f75-4844-92f3-4f7af3ec9294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:52.688944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4703bdc-6f75-4844-92f3-4f7af3ec9294 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:52.689242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4703bdc-6f75-4844-92f3-4f7af3ec9294 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:52.689503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4703bdc-6f75-4844-92f3-4f7af3ec9294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:52.697567 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4703bdc-6f75-4844-92f3-4f7af3ec9294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:52.699145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4703bdc-6f75-4844-92f3-4f7af3ec9294 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:52.708752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 759/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:13:56.611889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a46f-0645-4445-8df9-16077bb711d1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:56.612472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a46f-0645-4445-8df9-16077bb711d1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:56.612472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a46f-0645-4445-8df9-16077bb711d1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:56.613788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:13:56.619484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f46ce9-8607-4489-ad78-9d6e10ac0874 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:56.619899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f46ce9-8607-4489-ad78-9d6e10ac0874 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:56.620065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f46ce9-8607-4489-ad78-9d6e10ac0874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:56.639142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-79f46ce9-8607-4489-ad78-9d6e10ac0874 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:56.640370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-79f46ce9-8607-4489-ad78-9d6e10ac0874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:56.650129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:13:56.936534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a46f-0645-4445-8df9-16077bb711d1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:56.938228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6708a46f-0645-4445-8df9-16077bb711d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:57.017163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a46f-0645-4445-8df9-16077bb711d1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:57.019874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6515edb-11ad-48fd-bff6-93e5900dfafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:57.020475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515edb-11ad-48fd-bff6-93e5900dfafb service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:57.020753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515edb-11ad-48fd-bff6-93e5900dfafb service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:57.021031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515edb-11ad-48fd-bff6-93e5900dfafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:57.030381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6515edb-11ad-48fd-bff6-93e5900dfafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:57.032100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6515edb-11ad-48fd-bff6-93e5900dfafb service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:57.042213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 770/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:13:59.174071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be6e3b-2fa2-4691-bb49-fedc77a97405 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:59.174656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be6e3b-2fa2-4691-bb49-fedc77a97405 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:59.174656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be6e3b-2fa2-4691-bb49-fedc77a97405 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:59.176039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 10:13:59.181721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010d9cc-1636-4045-8fab-0699911c29ea None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:59.182159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010d9cc-1636-4045-8fab-0699911c29ea None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:59.182288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010d9cc-1636-4045-8fab-0699911c29ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:59.202281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6010d9cc-1636-4045-8fab-0699911c29ea None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:13:59.203521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6010d9cc-1636-4045-8fab-0699911c29ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:13:59.211447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:13:59.500474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be6e3b-2fa2-4691-bb49-fedc77a97405 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:13:59.511789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be6e3b-2fa2-4691-bb49-fedc77a97405 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:13:59.515323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784aabb-9b99-4064-850d-f38c308dfc96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:13:59.515906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784aabb-9b99-4064-850d-f38c308dfc96 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:13:59.516188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784aabb-9b99-4064-850d-f38c308dfc96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:13:59.516461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784aabb-9b99-4064-850d-f38c308dfc96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:13:59.525219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784aabb-9b99-4064-850d-f38c308dfc96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:13:59.526665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784aabb-9b99-4064-850d-f38c308dfc96 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:13:59.536283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 762/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:14:03.252445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429201-6abb-4690-b67c-8ba4029c2d8f None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:14:03.253010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429201-6abb-4690-b67c-8ba4029c2d8f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:14:03.253010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429201-6abb-4690-b67c-8ba4029c2d8f None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:14:03.254352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:14:03.260433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5417dd0-bfd8-4c9d-9f95-c103a626abbc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:14:03.260860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5417dd0-bfd8-4c9d-9f95-c103a626abbc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:14:03.261016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5417dd0-bfd8-4c9d-9f95-c103a626abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:14:03.268607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=81 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:14:03.280509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a5417dd0-bfd8-4c9d-9f95-c103a626abbc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:14:03.289496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a5417dd0-bfd8-4c9d-9f95-c103a626abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:14:03.290496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:14:03.592971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429201-6abb-4690-b67c-8ba4029c2d8f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:14:03.594844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7a429201-6abb-4690-b67c-8ba4029c2d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:14:03.665404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:14:03.678568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429201-6abb-4690-b67c-8ba4029c2d8f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:14:03.682263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ad919-8942-44b2-bdea-984006d24b0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:14:03.682874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ad919-8942-44b2-bdea-984006d24b0d service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:14:03.683157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ad919-8942-44b2-bdea-984006d24b0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:14:03.683462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ad919-8942-44b2-bdea-984006d24b0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:14:03.692407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ad919-8942-44b2-bdea-984006d24b0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:14:03.693902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ad919-8942-44b2-bdea-984006d24b0d service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:14:03.703651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 773/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:14:05.842300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd257f4-2b95-4cc9-b012-cfc100fdd1d7 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:14:05.842808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd257f4-2b95-4cc9-b012-cfc100fdd1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:14:05.842917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd257f4-2b95-4cc9-b012-cfc100fdd1d7 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:14:05.844418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 10:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:14:05.849470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050e739-5ad6-4c46-a123-03ad8a4aa143 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:14:05.849922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050e739-5ad6-4c46-a123-03ad8a4aa143 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:14:05.850014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050e739-5ad6-4c46-a123-03ad8a4aa143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:14:05.870908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b050e739-5ad6-4c46-a123-03ad8a4aa143 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:14:05.872260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b050e739-5ad6-4c46-a123-03ad8a4aa143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:14:05.880241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:14:06.182594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd257f4-2b95-4cc9-b012-cfc100fdd1d7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:14:06.184371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7fd257f4-2b95-4cc9-b012-cfc100fdd1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:14:06.198050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd257f4-2b95-4cc9-b012-cfc100fdd1d7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:14:06.201723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1185d6-1539-4262-ae91-ced72d07c1a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:14:06.202319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1185d6-1539-4262-ae91-ced72d07c1a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:14:06.202602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1185d6-1539-4262-ae91-ced72d07c1a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:14:06.202878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1185d6-1539-4262-ae91-ced72d07c1a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:14:06.211644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1185d6-1539-4262-ae91-ced72d07c1a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:14:06.213050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1185d6-1539-4262-ae91-ced72d07c1a7 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:14:06.222592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 765/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:14:15.865338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=11 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:14:16.218152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=8 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:30.753924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:30.755201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:30.755201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:30.777686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:30.779182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:30.788361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 775/1540] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:30.796837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:30.797411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:30.797411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:30.828185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:30.829939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:30.848007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 766/1541] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:30.855922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:30.866503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3195647-8bb4-4ab9-b704-3169190ffb6e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:30.875964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:30.876826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:30.877313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:30.877781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:30.878638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:30.879141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:30.881015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:30.897357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 776/1542] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:30.905657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:30.920192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4419-81e3-45a9-abe9-ae262e57bc5d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:30.929544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb59ced-72e5-4c6e-8345-f23ccf4c804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:30.930356 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb59ced-72e5-4c6e-8345-f23ccf4c804b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:30.930794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb59ced-72e5-4c6e-8345-f23ccf4c804b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:30.931262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb59ced-72e5-4c6e-8345-f23ccf4c804b admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:30.932081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb59ced-72e5-4c6e-8345-f23ccf4c804b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:30.934146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb59ced-72e5-4c6e-8345-f23ccf4c804b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.075919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 767/1543] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:30 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:31.079875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.092333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0672c7-aa86-400e-aa56-58e74c23e38a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.183012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.183012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.183012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.183012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.183465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.183911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.206674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa68aa3-35a1-4a7c-b739-cdf5926aa5de None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.207397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa68aa3-35a1-4a7c-b739-cdf5926aa5de None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.208723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa68aa3-35a1-4a7c-b739-cdf5926aa5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.234814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ffa68aa3-35a1-4a7c-b739-cdf5926aa5de None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.236509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ffa68aa3-35a1-4a7c-b739-cdf5926aa5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.254800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 768/1544] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.264810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.265486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.265999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.294594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.297784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.313444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 769/1545] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.318731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.332387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 777/1546] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:31.332829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d1a5a-0196-4da2-8d09-a2d595ac2349 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.336175 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.347875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.348600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64aa79-2979-4189-84b6-415a32bba459 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.349158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.349346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.349583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.350887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:31.351323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.352352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:15:31.352352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:15:31.352352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:15:31.353252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.359350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.362996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.362996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.362996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.362996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.362996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.383587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 770/1547] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:15:31.385907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 778/1548] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:31.404314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.412363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.427870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7363-7f26-4805-ad56-683743c764a6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.432184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778df15-62d9-4268-b585-6eee3a1c6927 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.441460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.442152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.442436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.442710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.443438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.444644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:15:31.444644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:15:31.444644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:15:31.446123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eea49-8540-43cb-a590-e80d4a268d64 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.464064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75ab5-8e33-4d31-a98e-50f2a58a4050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.464708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75ab5-8e33-4d31-a98e-50f2a58a4050 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.464991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75ab5-8e33-4d31-a98e-50f2a58a4050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.465265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75ab5-8e33-4d31-a98e-50f2a58a4050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.465922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75ab5-8e33-4d31-a98e-50f2a58a4050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.467707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75ab5-8e33-4d31-a98e-50f2a58a4050 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.489820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 771/1549] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:31.493041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.493041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.493041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.529196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.534952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.542596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 772/1550] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.543752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.556828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885c54a-beef-4466-aeb8-65b607598f9f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.567236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84e104-32c4-4666-a451-437358ad7186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.568400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e104-32c4-4666-a451-437358ad7186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.570048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 779/1551] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/projects => generated 399 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:31.570737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e104-32c4-4666-a451-437358ad7186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.571098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e104-32c4-4666-a451-437358ad7186 admin admin] PATH_INFO: `/v3/projects/30beae855402476b9e1e542be6cc7e88/users/ca6855123d0740aba728a2379419cc95/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.573056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347e76b-ddb0-4980-adce-94148da0db74 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.573348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347e76b-ddb0-4980-adce-94148da0db74 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.573626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347e76b-ddb0-4980-adce-94148da0db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.595591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e104-32c4-4666-a451-437358ad7186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30beae855402476b9e1e542be6cc7e88, user_id=ca6855123d0740aba728a2379419cc95, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.598951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e104-32c4-4666-a451-437358ad7186 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.604106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4347e76b-ddb0-4980-adce-94148da0db74 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.605330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4347e76b-ddb0-4980-adce-94148da0db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.667258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 780/1552] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.667258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e678-25d8-4f41-a248-4653e55daee3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.667258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e678-25d8-4f41-a248-4653e55daee3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.667258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e678-25d8-4f41-a248-4653e55daee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.675256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 773/1553] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:31 2022] PUT /identity/v3/projects/30beae855402476b9e1e542be6cc7e88/users/ca6855123d0740aba728a2379419cc95/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:31.676886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.677957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.678707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.693745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5863e678-25d8-4f41-a248-4653e55daee3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.695016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5863e678-25d8-4f41-a248-4653e55daee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.700306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.702364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.730861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:31.759271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 781/1554] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.760409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60088ca2-383f-4fe0-9a50-0d29a46692cd None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.760743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60088ca2-383f-4fe0-9a50-0d29a46692cd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.760971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60088ca2-383f-4fe0-9a50-0d29a46692cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.764381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 774/1555] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.765490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.775621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37eefb4-fbf3-4836-afa7-f85191fe5376 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.780647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.781243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.781520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.781789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.782704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.783437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:15:31.783437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:15:31.783437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:15:31.784652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.786472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-60088ca2-383f-4fe0-9a50-0d29a46692cd None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.787716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-60088ca2-383f-4fe0-9a50-0d29a46692cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.795867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 782/1556] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.797052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41538d-fc6b-4f16-b7bb-60cef8a673c8 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.797334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41538d-fc6b-4f16-b7bb-60cef8a673c8 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.797606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41538d-fc6b-4f16-b7bb-60cef8a673c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.818752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ad41538d-fc6b-4f16-b7bb-60cef8a673c8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.819968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ad41538d-fc6b-4f16-b7bb-60cef8a673c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.827881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 783/1557] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.829023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f386-f0a7-4430-b923-b91c1b6c7444 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.829306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f386-f0a7-4430-b923-b91c1b6c7444 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.829579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f386-f0a7-4430-b923-b91c1b6c7444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.849003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6858f386-f0a7-4430-b923-b91c1b6c7444 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.850117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6858f386-f0a7-4430-b923-b91c1b6c7444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.858191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 784/1558] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.859426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b9e3-136f-4e5b-9ed5-e9cf1110c5e5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.859738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b9e3-136f-4e5b-9ed5-e9cf1110c5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.860010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b9e3-136f-4e5b-9ed5-e9cf1110c5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.881027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ef34b9e3-136f-4e5b-9ed5-e9cf1110c5e5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.882214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ef34b9e3-136f-4e5b-9ed5-e9cf1110c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.889998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 785/1559] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.891233 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa9148-3219-4da0-b253-8060c686b6b4 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.891520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa9148-3219-4da0-b253-8060c686b6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.891796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa9148-3219-4da0-b253-8060c686b6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.894390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 775/1560] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:31.895932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.907337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd4b05-0d29-4a5c-ae0f-1c8b20fd2147 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.912337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:31.912957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.913188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-dfaa9148-3219-4da0-b253-8060c686b6b4 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.913314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.913498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.914146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:31.914452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:31.914566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dfaa9148-3219-4da0-b253-8060c686b6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.916090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c745c00-8214-4adf-9f6b-f73a7e6283f1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:31.922787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 786/1561] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.923985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc955b93-2df9-460f-b287-472f9c5a436d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.924267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc955b93-2df9-460f-b287-472f9c5a436d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.924542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc955b93-2df9-460f-b287-472f9c5a436d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.935986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 776/1562] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:31.935986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c460960-00fb-4f07-840f-84da5d970293 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.935986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c460960-00fb-4f07-840f-84da5d970293 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.935986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c460960-00fb-4f07-840f-84da5d970293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.944745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bc955b93-2df9-460f-b287-472f9c5a436d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.945955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bc955b93-2df9-460f-b287-472f9c5a436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.953155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4c460960-00fb-4f07-840f-84da5d970293 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.953912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 787/1563] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.954374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4c460960-00fb-4f07-840f-84da5d970293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.955313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.955596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.955866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.964597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 777/1564] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.965949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:31.966238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:31.966540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:31.975957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.977147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.985012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 788/1565] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.985772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:31.986285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:31.987609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:31.994794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d22d3-31ed-4f89-872a-709c20bf7ab0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:31.997005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 778/1566] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:31.999465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.000044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.000323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.000593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.001114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.001229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.002390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:32.002791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.013338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 789/1567] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:32.013977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.016897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:32.023843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239948c1-0777-4f0d-b77b-ba115ae20395 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.028344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.030558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.030558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.030558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.030558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:32.030953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.032259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.044254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 790/1568] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:32.045137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.054800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e300f-479c-4668-a26a-462706140dcf admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.060071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.060595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.060878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.061153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.061779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.063723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.069418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d80b14-99c4-4c7f-bba6-3d1593baac4b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.073924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.074162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.074447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.074723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.084675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.086137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.095714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 779/1569] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:32.096489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.105934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6e23c-a187-40b7-99db-8da17f1ab80a service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.110623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.115418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.115418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.115418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.115418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:32.115418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.115418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.127141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 780/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:32.134182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.136905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 791/1571] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/projects => generated 393 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.137663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.138410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd615-3c5c-4383-a0e1-a6ac1b7204fd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.200879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.201254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75868cb5-3952-415e-898a-6d7700a19824 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.201507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.201791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.202071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.202718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:32.203067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.204689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.208466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.208466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.208466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.208466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.209124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:32.209420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.211012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.227055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 781/1572] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:32.227055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.227991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 792/1573] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:32.227991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.240611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba67e43-939c-49c8-83c3-1f6c016f013a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.241766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee227ac-8c49-4521-8c7a-ee0d76ae24f5 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.245683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.246330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.246616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.246884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.247557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:32.247699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.247890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.248658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.249018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.249369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.249523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.250146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.251875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.263080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 782/1574] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:32.264112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.264549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 793/1575] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:32.265352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.276344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571e0be-ab97-4533-bc89-80e511272035 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.280640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba56836-187f-4431-b78d-255001a5cecf admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.281800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.282468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.282758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.283031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.283697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.286803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.287525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.288362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.288835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.289304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.290134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.292086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.383949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 783/1576] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.384155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 794/1577] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.384281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.384390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.402313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:32.461726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299605-38d8-4f56-9cda-860ff9a4d7d7 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.462524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f831e-7b09-4b78-94f4-548c470fceda admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.467916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.468222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.468567 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.468842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.468966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.469120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.469255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.469437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.469759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.470018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.471742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.471950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.481036 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:32.572860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 784/1578] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.572860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.573842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 795/1579] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.574730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.585373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ce0c-b1af-456b-9faf-0c95d543c5fb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.653224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb192aa-ca51-4566-9150-1bca911edbfe admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.656211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.657581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.657581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.657581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.658013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.660410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.672768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.673925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.674467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.675101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.676071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.679240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.749707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 796/1580] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.749707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.761859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 785/1581] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/projects => generated 367 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.761859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.828027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7874c-9bb8-42ad-a12f-6352cfbd7958 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.828912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51438ac7-31ac-46f4-a0e8-ee6fe703ee0f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.833785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.834495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.834626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.834818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.835206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.835473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.835846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.835986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.836134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] PATH_INFO: `/v3/projects/bab9fc25a2364ae79578ccc87de8bc35/users/af64f2a8387e4996bd91f9aadfd792d4/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.838168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.855055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab9fc25a2364ae79578ccc87de8bc35, user_id=af64f2a8387e4996bd91f9aadfd792d4, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.856179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:15:32.856179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:15:32.856179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:15:32.858118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.951278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 786/1582] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:32.952047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:32.961568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790632-3bde-4b5d-affd-17cf199c473d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:32.966212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:32.966782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:32.967070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:32.967375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:32.967991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:32.969881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:32.975428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:32.983006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 797/1583] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:32 2022] PUT /identity/v3/projects/bab9fc25a2364ae79578ccc87de8bc35/users/af64f2a8387e4996bd91f9aadfd792d4/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:32.983900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.050415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256442fe-b1f7-4703-9210-9ca43b3f424c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.056172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.056788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.057139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.057421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.058073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.059720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.070431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 798/1584] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.071565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.075373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 787/1585] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:32 2022] POST /identity/v3/users => generated 302 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:33.075373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.082192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21addb-846c-4f51-8185-b1c12947f634 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.082894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086398f0-e203-4533-816d-a7a2a3bdf1da admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.087505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.087639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.088131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.088257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.088414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.088514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.088687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.088796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.089314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.089422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.091254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.091362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.180640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 799/1586] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:33 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:33.183756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 788/1587] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:33 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:33.183756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.184903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.194372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995923b-61bc-4537-b4e3-bed0ac845adb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.194948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71997-c65a-4f31-86fa-0091fd836c89 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.199122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687263b-c498-4081-aebd-a89bf3bb75c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.199636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.199773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687263b-c498-4081-aebd-a89bf3bb75c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.200010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687263b-c498-4081-aebd-a89bf3bb75c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.200229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.200344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687263b-c498-4081-aebd-a89bf3bb75c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.200529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.200788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.200950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687263b-c498-4081-aebd-a89bf3bb75c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.201478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.202535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687263b-c498-4081-aebd-a89bf3bb75c1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.203349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.212699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 789/1588] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.217169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.217169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.217169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.233064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:33.234254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:33.247971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 790/1589] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:33.247971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.255352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100be12-7ca6-4de7-bfd7-396fe9dcf6eb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.260361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.261019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.261303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.261578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.262231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.263923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.269589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=13,keystone:SELECT=87 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:33.274717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 791/1590] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.275617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.286326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0351a8-387a-4585-b23b-83bc5064fd6e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.291221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.291819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.292101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.292401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.293041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.294603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.305867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 792/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.306690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.319383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d61495-08fd-4283-9c0f-ee8ccbf9a783 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.324772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.325394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.325701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.326083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.326902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.328536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.332484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 800/1592] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:33 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:33.332978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.344334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 793/1593] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.345073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924a28-3392-42b5-84c0-bb4bf781a150 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.345633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.351735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.352365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.352623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.365578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.365578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.365578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.365997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c86bc-2470-427f-9238-61d94672ba3a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.370569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.371313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.371664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.372629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.373351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.376858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 801/1594] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.376858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.377092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.392619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 794/1595] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.392619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.406566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd39a91-8909-40e1-9e71-29682c25d3fd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.413199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.426896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.426896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.426896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] PATH_INFO: `/v3/projects/602cd87d731241e3ad12032e2dd67c1b/users/1cc3e06e1b1440c18c0fb902beeed27d/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.431974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=602cd87d731241e3ad12032e2dd67c1b, user_id=1cc3e06e1b1440c18c0fb902beeed27d, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.434971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.449333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b0a29-de6f-42b2-8e49-48e9aed54d6b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.453326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.453869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.454145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.454414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.463569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.464995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.475381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 802/1596] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:33.476198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.485655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0a8f1-7175-4cf6-8e46-9080ce5b3d15 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.491194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.491809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.492090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.492362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.493018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.494617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.507724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 803/1597] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.509757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.521385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa358a6-ccf7-45f7-84b0-2f7adf6d104c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.523590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 795/1598] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:33 2022] PUT /identity/v3/projects/602cd87d731241e3ad12032e2dd67c1b/users/1cc3e06e1b1440c18c0fb902beeed27d/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:33.524643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.578998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:33.601555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.602576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.603096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.603627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.604495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.607278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.611155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb202f6-a405-482c-b5c5-d2d66a83bed3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.619675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.620528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.620731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.621153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.621716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 804/1599] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.621839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.623018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.624060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.636895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 796/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.637184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156396c6-5024-4f40-b4e4-6b8113f0e5cb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.637894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.643402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.644287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.645725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.646206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.648024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.650222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.653063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f940c-e8ae-4005-bdf9-6bfc845c6f2e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.662052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.662872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.663631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.664069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] PATH_INFO: `/v3/projects/b45075d7ec024b3a907d11225942fbee/users/c069999293a847cdb11650045e22576c/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.665764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 805/1601] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.666881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.685126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba9112-e425-4f5b-8437-b85a5360dba1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.686429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fb843-2586-4c1a-8551-7ced0f772e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.686711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45075d7ec024b3a907d11225942fbee, user_id=c069999293a847cdb11650045e22576c, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.687353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fb843-2586-4c1a-8551-7ced0f772e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.687709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fb843-2586-4c1a-8551-7ced0f772e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.688051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fb843-2586-4c1a-8551-7ced0f772e14 admin admin] PATH_INFO: `/v3/projects/78ee81448f0f4e68956d363775bcc01c/users/8f3c2d505a2c45c6a0b23fcafc19cd5d/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.689808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.713302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fb843-2586-4c1a-8551-7ced0f772e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ee81448f0f4e68956d363775bcc01c, user_id=8f3c2d505a2c45c6a0b23fcafc19cd5d, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.713779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fb843-2586-4c1a-8551-7ced0f772e14 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.805931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 806/1603] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:33 2022] PUT /identity/v3/projects/78ee81448f0f4e68956d363775bcc01c/users/8f3c2d505a2c45c6a0b23fcafc19cd5d/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:33.805931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.805931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.805931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.807400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 797/1603] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:33 2022] PUT /identity/v3/projects/b45075d7ec024b3a907d11225942fbee/users/c069999293a847cdb11650045e22576c/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:33.808830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.838246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:33.840324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:33.890826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af689827-7d1d-46b3-afe5-d85d5d02a396 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.898816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.898816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.898816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.899111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.899853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.901464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.916026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 798/1604] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:33.916026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.927339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d515-5673-4563-a44c-f6638e05942d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.932466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13452713-846e-4e95-b335-105a30035213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.933142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452713-846e-4e95-b335-105a30035213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.933429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452713-846e-4e95-b335-105a30035213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.933711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452713-846e-4e95-b335-105a30035213 admin admin] PATH_INFO: `/v3/projects/66658bb98ea64e319ed948b8f279ed5f/users/4067d48268f84854af4d672ab3f9dc65/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.937322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 807/1605] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:33.938269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:33.950544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6dafe-122a-4a49-8c8f-e79e6f9a4055 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:33.955358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:33.961924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bb2f8-190e-4376-99a1-16fccd802cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:33.961924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bb2f8-190e-4376-99a1-16fccd802cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:33.961924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bb2f8-190e-4376-99a1-16fccd802cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:33.961924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bb2f8-190e-4376-99a1-16fccd802cad admin admin] PATH_INFO: `/v3/projects/1d34f4c2ba414b909e13cc8985491a4b/users/996523e7efb0496587c599c8bcb2c584/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:33.967629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13452713-846e-4e95-b335-105a30035213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66658bb98ea64e319ed948b8f279ed5f, user_id=4067d48268f84854af4d672ab3f9dc65, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.970916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13452713-846e-4e95-b335-105a30035213 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:33.980379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bb2f8-190e-4376-99a1-16fccd802cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d34f4c2ba414b909e13cc8985491a4b, user_id=996523e7efb0496587c599c8bcb2c584, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:33.985402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bb2f8-190e-4376-99a1-16fccd802cad admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.060193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 799/1606] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:33 2022] PUT /identity/v3/projects/66658bb98ea64e319ed948b8f279ed5f/users/4067d48268f84854af4d672ab3f9dc65/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:34.061683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.061967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.062239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.064192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 808/1607] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:33 2022] PUT /identity/v3/projects/1d34f4c2ba414b909e13cc8985491a4b/users/996523e7efb0496587c599c8bcb2c584/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:34.065684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.066075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.066502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.096401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:34.097664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:34.106456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] MFA Rules not processed for user `8f3c2d505a2c45c6a0b23fcafc19cd5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:34.107825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:34.193725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 809/1608] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:34.194726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:34.206925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:34.233648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 800/1609] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:34.234258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:34.287608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8dcd4-b03a-417e-8ad5-aa24b7804ea6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:34.295838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:34.296786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.297261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.297656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] PATH_INFO: `/v3/projects/87d1f004835544b6bbbfe6f90231187b/users/9e68f00088ac4ec4a145eb4d39a8b97f/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.304338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea490e54-6930-4512-8049-65f4012f5615 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:34.308389 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6292d3-7ae4-490d-a1d7-e8d0f4515333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:34.309074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6292d3-7ae4-490d-a1d7-e8d0f4515333 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.309352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6292d3-7ae4-490d-a1d7-e8d0f4515333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.309679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6292d3-7ae4-490d-a1d7-e8d0f4515333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.321868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d1f004835544b6bbbfe6f90231187b, user_id=9e68f00088ac4ec4a145eb4d39a8b97f, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:34.326082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6292d3-7ae4-490d-a1d7-e8d0f4515333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:34.326652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.327806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6292d3-7ae4-490d-a1d7-e8d0f4515333 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.340321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 801/1610] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:34.341245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5cb8-a49f-4042-9d17-3db79c013e34 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.341536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5cb8-a49f-4042-9d17-3db79c013e34 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.341812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5cb8-a49f-4042-9d17-3db79c013e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.364512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-448d5cb8-a49f-4042-9d17-3db79c013e34 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:34.366455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-448d5cb8-a49f-4042-9d17-3db79c013e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:34.375035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 802/1611] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:34.376047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.376333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.378403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.404973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 810/1612] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:34 2022] PUT /identity/v3/projects/87d1f004835544b6bbbfe6f90231187b/users/9e68f00088ac4ec4a145eb4d39a8b97f/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:34.405854 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:34.434823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] MFA Rules not processed for user `996523e7efb0496587c599c8bcb2c584`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:34.436373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:34.528560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b2989-b9a6-4d8a-a583-af058a370548 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:34.528560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:34.528560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.528560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.529157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.529157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:34.534487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.607216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 803/1613] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:34.607216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:34.625192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:34.686135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 811/1614] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/projects => generated 405 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:34.687006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:34.706258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab941e-c2f3-4ddc-8e3a-37c407c9104b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:34.770857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dacfe-a993-4a47-9dc6-41bd02794545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:34.771618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dacfe-a993-4a47-9dc6-41bd02794545 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.771976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dacfe-a993-4a47-9dc6-41bd02794545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.772306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dacfe-a993-4a47-9dc6-41bd02794545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.774246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d74660-40f7-482b-97eb-008f9bd4bd79 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:34.781938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:34.785691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.786665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.787194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.839467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:34.861487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dacfe-a993-4a47-9dc6-41bd02794545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:34.862939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dacfe-a993-4a47-9dc6-41bd02794545 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.875837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 804/1615] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:34.877079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.877423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.877704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.902844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:34.904553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.911096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] MFA Rules not processed for user `9e68f00088ac4ec4a145eb4d39a8b97f`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:34.913017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:34.918657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 812/1616] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:34.919822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:34.930365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72602b57-104f-4d0d-b50d-cc21ccedb8d8 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:34.936479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:34.937514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:34.937872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:34.938244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:34.939028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:34.941297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:34.995518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 805/1617] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:34.995518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.009088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884ad60-c197-4713-8da6-3d7614f0aadd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.016969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.017612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.017936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.018293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.018948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.021513 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.052132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 813/1618] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:35.053570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.118498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 806/1619] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:34 2022] POST /identity/v3/users => generated 309 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:35.122240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.152286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3edd6f-4c2d-4e43-9e7e-532433612f8c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.170859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949dc80-789f-49c7-b209-3bc09a5cf84b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.183596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.184922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.185585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.188009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.266517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:35.281445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.282370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.282691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.283078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.283894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.287972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.311080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.312656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.323561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 814/1620] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:35.324488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.340236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096309a0-a424-49ca-99d0-183aa6f9ed29 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.346177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.346800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.347087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.347474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.348831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.350514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.369291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 815/1621] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:35.373864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.400471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea9db5-c087-4b08-bc74-c2934950101c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.409455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.411358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.412302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.412752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] PATH_INFO: `/v3/projects/dd47ed6fc64a46c4a7386cb7025f9d06/users/4cec7ff60e9d406883219c6834e879a0/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.435727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 807/1622] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:35 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:35.436730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.445506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd47ed6fc64a46c4a7386cb7025f9d06, user_id=4cec7ff60e9d406883219c6834e879a0, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.451013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.466237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccf10f-f6a3-40a0-9014-0d92ac0055de admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.480685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.481337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.481781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.482063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.482721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.484607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.500266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 808/1623] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:35.505674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.520433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9f21d-76de-4a6a-add8-166a799d0fdc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.527132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.533464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.533464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.533464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] PATH_INFO: `/v3/projects/09e4028cba7c4eac9907b6e7ae29b584/users/2f4aa6408e764738a2d556a33172d0b5/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.558003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e4028cba7c4eac9907b6e7ae29b584, user_id=2f4aa6408e764738a2d556a33172d0b5, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.563347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.586984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 816/1624] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:35 2022] PUT /identity/v3/projects/dd47ed6fc64a46c4a7386cb7025f9d06/users/4cec7ff60e9d406883219c6834e879a0/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:15:35.591136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.641643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:35.679475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 809/1625] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:35 2022] PUT /identity/v3/projects/09e4028cba7c4eac9907b6e7ae29b584/users/2f4aa6408e764738a2d556a33172d0b5/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:35.680411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.707759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb08ce5-5bab-4044-ba4e-576167b130e4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.858252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.859050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.859153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.859446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.860099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.861741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.865967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c0534-0adc-4bee-937c-758eae6b5a1d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.879443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 817/1626] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:15:35.883003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.883401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.884103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.884344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.884613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.885467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.886941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.896026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f642702-11d0-4f40-a82c-2b9153382312 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.899201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 810/1627] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:35.902850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c68e8-d3bc-4f2a-b842-18d2b9c46cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:35.907323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c68e8-d3bc-4f2a-b842-18d2b9c46cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.907323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c68e8-d3bc-4f2a-b842-18d2b9c46cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.907323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c68e8-d3bc-4f2a-b842-18d2b9c46cfd admin admin] PATH_INFO: `/v3/projects/dd47ed6fc64a46c4a7386cb7025f9d06/users/4cec7ff60e9d406883219c6834e879a0/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:35.907323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c68e8-d3bc-4f2a-b842-18d2b9c46cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd47ed6fc64a46c4a7386cb7025f9d06, user_id=4cec7ff60e9d406883219c6834e879a0, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:35.908332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:35.911250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c68e8-d3bc-4f2a-b842-18d2b9c46cfd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:35.922693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9bee2-82c5-4495-9dea-cc929bf4038b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:35.949124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 818/1628] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:35 2022] PUT /identity/v3/projects/dd47ed6fc64a46c4a7386cb7025f9d06/users/4cec7ff60e9d406883219c6834e879a0/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:15:35.954097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649efc-6e3b-4582-88a0-eb786c80a35f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:35.956390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649efc-6e3b-4582-88a0-eb786c80a35f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:35.956643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649efc-6e3b-4582-88a0-eb786c80a35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.021993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a1649efc-6e3b-4582-88a0-eb786c80a35f None None] MFA Rules not processed for user `af64f2a8387e4996bd91f9aadfd792d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:36.025287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a1649efc-6e3b-4582-88a0-eb786c80a35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:36.041647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307b5ce-4b6d-4659-9da0-a85b30d36f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:36.042497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307b5ce-4b6d-4659-9da0-a85b30d36f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.042782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307b5ce-4b6d-4659-9da0-a85b30d36f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.043155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307b5ce-4b6d-4659-9da0-a85b30d36f73 admin admin] PATH_INFO: `/v3/projects/09e4028cba7c4eac9907b6e7ae29b584/users/2f4aa6408e764738a2d556a33172d0b5/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.048885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307b5ce-4b6d-4659-9da0-a85b30d36f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e4028cba7c4eac9907b6e7ae29b584, user_id=2f4aa6408e764738a2d556a33172d0b5, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:36.051944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307b5ce-4b6d-4659-9da0-a85b30d36f73 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:36.087441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 811/1629] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:35 2022] PUT /identity/v3/projects/09e4028cba7c4eac9907b6e7ae29b584/users/2f4aa6408e764738a2d556a33172d0b5/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:36.087441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.087441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.087441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.119446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 819/1630] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:36.119446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.119446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.119446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.144480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] MFA Rules not processed for user `4cec7ff60e9d406883219c6834e879a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:36.146483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:36.156442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] MFA Rules not processed for user `2f4aa6408e764738a2d556a33172d0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:36.158182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:36.282778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 820/1631] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:36.285210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:36.299362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:36.318427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 812/1632] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:36.320258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:36.398968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813fada-55ea-469c-ac5f-13d95e69e6c0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:36.405920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:36.408116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.408515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.408890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.409643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:36.412633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:36.433295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f446e-c10d-4e1d-89a5-35c79b5cf774 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:36.758087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:36.758087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.758087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.758087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.770289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:36.770289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:36.770289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 813/1633] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:36.771595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:36.771595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 821/1634] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:36 2022] POST /identity/v3/projects => generated 405 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:36.772446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:36.772446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d89f40-4d3c-477a-a2f4-14d5a9acf5ae service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:36.773157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:36.773340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:36.773340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.773340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.773340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:36.775448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:36.781672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:36.902360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c9867-0043-457c-88fc-85576ac93488 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:36.919332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:36.924740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:36.928555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:36.929015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:37.018969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 814/1635] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:36 2022] POST /identity/v3/users => generated 309 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:37.023404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:37.042627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:37.045132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:37.045249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a582f8-b111-4abe-acd0-07b03dce042b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:37.061731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:37.062916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 822/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:37.063950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:37.064239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:37.064516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:37.065179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:37.067487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:37.106933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 815/1637] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:37.108205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:37.128962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800833-67af-4ec4-a5c4-b45d4acb2f5f service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:37.146517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:37.147410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:37.149290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:37.150244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] PATH_INFO: `/v3/projects/cb6c730fed914bb3906059b9b58b1195/users/feed6fb6fc4e4558ad762f9468bff195/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:37.208753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6c730fed914bb3906059b9b58b1195, user_id=feed6fb6fc4e4558ad762f9468bff195, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:37.217121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:37.319849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 823/1638] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:37 2022] PUT /identity/v3/projects/cb6c730fed914bb3906059b9b58b1195/users/feed6fb6fc4e4558ad762f9468bff195/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:37.324015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:37.338503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb0f96-e367-41be-b6ec-c5e2ca8c511d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:37.434166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:37.524367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:37.551888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:37.552562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:37.552800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:37.555210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:37.555898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:37.561573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:37.611566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 816/1639] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:37.620342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:37.638974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6962abd-1bef-4e15-a542-2bfde715b5de admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:37.653126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:37.654267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:37.654549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:37.655830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] PATH_INFO: `/v3/projects/cb6c730fed914bb3906059b9b58b1195/users/feed6fb6fc4e4558ad762f9468bff195/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:37.661744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6c730fed914bb3906059b9b58b1195, user_id=feed6fb6fc4e4558ad762f9468bff195, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:37.667730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:37.694622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 817/1640] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:37 2022] PUT /identity/v3/projects/cb6c730fed914bb3906059b9b58b1195/users/feed6fb6fc4e4558ad762f9468bff195/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:37.713327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0175d-0673-4a78-a7d9-4b37cd8f40eb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:37.866916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:37.867671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:37.868018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:37.872286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] PATH_INFO: `/v3/users/8f3c2d505a2c45c6a0b23fcafc19cd5d` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:37.872286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3c2d505a2c45c6a0b23fcafc19cd5d)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:37.872286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: WARNING py.warnings [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:15:37.872286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: warnings.warn(msg) Sep 13 10:15:37.872286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: Sep 13 10:15:37.872286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:37.922248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:38.164482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3c2d505a2c45c6a0b23fcafc19cd5d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.220560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:38.239722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e876-e8f4-4fbd-ba41-6ba662194c8c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:38.248418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:38.249301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:38.249444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:38.249941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] PATH_INFO: `/v3/users/996523e7efb0496587c599c8bcb2c584` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:38.251368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=996523e7efb0496587c599c8bcb2c584)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:38.253506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:38.270714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3c2d505a2c45c6a0b23fcafc19cd5d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.271910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3c2d505a2c45c6a0b23fcafc19cd5d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.272217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3c2d505a2c45c6a0b23fcafc19cd5d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.296045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8cb9f0e0-a7ba-4a3a-a6b1-020cd770e032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3c2d505a2c45c6a0b23fcafc19cd5d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.353643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 824/1641] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:37 2022] DELETE /identity/v3/users/8f3c2d505a2c45c6a0b23fcafc19cd5d => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:38.355305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:38.355605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:38.355889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:38.401571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:38.402861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:38.505993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '996523e7efb0496587c599c8bcb2c584'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.526745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 825/1642] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:38.548517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:38.550303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:38.600305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '996523e7efb0496587c599c8bcb2c584'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.601210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '996523e7efb0496587c599c8bcb2c584'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.602723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '996523e7efb0496587c599c8bcb2c584'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.615512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bc68740b-e89f-4e98-b247-06a17d8fc629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '996523e7efb0496587c599c8bcb2c584'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:38.658468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 818/1643] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:38 2022] DELETE /identity/v3/users/996523e7efb0496587c599c8bcb2c584 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:38.663699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:38.664263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:38.664727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:38.678764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e68ec-46fd-4cbf-b8c9-f0702c2015b7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:38.712938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:38.714322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:38.790263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:38.790911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:38.791521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:38.791521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:38.802174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:38.846235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 819/1644] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:15:38.882404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:38.885824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:38.905676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4816460-45c6-4857-bf3c-6c6c934fdadc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:38.911852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba2e61-bac1-477c-a76e-45778a09fb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:38.912695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba2e61-bac1-477c-a76e-45778a09fb9e service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:38.913085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba2e61-bac1-477c-a76e-45778a09fb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:38.913876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba2e61-bac1-477c-a76e-45778a09fb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:38.926653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba2e61-bac1-477c-a76e-45778a09fb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:38.928536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba2e61-bac1-477c-a76e-45778a09fb9e service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:38.935541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:38.944366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 820/1645] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:38.955233 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:38.956730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bf791-8d00-4c2e-91e1-06e24b9034ff service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:38.976999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 826/1646] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:15:39.492177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:39.657694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be3ac-4acf-4d8b-9702-6066fa986095 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:39.658238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be3ac-4acf-4d8b-9702-6066fa986095 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:39.658238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be3ac-4acf-4d8b-9702-6066fa986095 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:39.660134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 821/1647] 10.222.0.62 () {54 vars in 962 bytes} [Tue Sep 13 10:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:39.672617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f240b-d474-4656-84e5-481e0bb84ace None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:39.673225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f240b-d474-4656-84e5-481e0bb84ace None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:39.673715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f240b-d474-4656-84e5-481e0bb84ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:39.740559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-493f240b-d474-4656-84e5-481e0bb84ace None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:39.743748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-493f240b-d474-4656-84e5-481e0bb84ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:39.842884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 827/1648] 10.222.0.62 () {58 vars in 1059 bytes} [Tue Sep 13 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:39.844090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612956d-09ee-4ef6-a0f7-ff600c24a599 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:39.844442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612956d-09ee-4ef6-a0f7-ff600c24a599 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:39.844726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612956d-09ee-4ef6-a0f7-ff600c24a599 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:39.846162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 822/1649] 10.222.0.62 () {54 vars in 912 bytes} [Tue Sep 13 10:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:39.855548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:39.855840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:39.856125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:39.911419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:39.912994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:39.936077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 828/1650] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:39.945256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612956d-09ee-4ef6-a0f7-ff600c24a599 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:39.947526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f612956d-09ee-4ef6-a0f7-ff600c24a599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:40.028250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:40.030404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:40.061461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9c52-0619-466a-a1d8-6d5b1877977e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:40.079133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:40.079839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:40.080125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:40.080404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] PATH_INFO: `/v3/projects/1d34f4c2ba414b909e13cc8985491a4b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:40.086618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d34f4c2ba414b909e13cc8985491a4b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:40.090060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:40.102564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612956d-09ee-4ef6-a0f7-ff600c24a599 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:40.114185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:40.114825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:40.115110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:40.115421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:40.173226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:40.174700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:40.210673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 823/1651] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:40.352047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d34f4c2ba414b909e13cc8985491a4b', 'request_id': 'req-e355a324-918b-4ff8-9fd0-ae7b76efe874'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:40.372135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d34f4c2ba414b909e13cc8985491a4b', 'request_id': 'req-e355a324-918b-4ff8-9fd0-ae7b76efe874'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:40.475347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d34f4c2ba414b909e13cc8985491a4b', 'request_id': 'req-e355a324-918b-4ff8-9fd0-ae7b76efe874'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:40.498285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] The token cache is being invalidate because project 1d34f4c2ba414b909e13cc8985491a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:40.498716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:40.499566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:40.501283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 829/1652] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:40 2022] DELETE /identity/v3/projects/1d34f4c2ba414b909e13cc8985491a4b => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:40.505975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:40.523323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d97e36-4bc6-4a0a-940f-33eb7efa4ca8 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:40.620617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:40.621722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:40.622209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:40.622712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] PATH_INFO: `/v3/users/2f4aa6408e764738a2d556a33172d0b5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:40.624286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4aa6408e764738a2d556a33172d0b5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:40.627317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:40.890130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:40.891759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:40.917031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355a324-918b-4ff8-9fd0-ae7b76efe874 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:40.930999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:40.931715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:40.932022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:40.932399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] PATH_INFO: `/v3/projects/78ee81448f0f4e68956d363775bcc01c` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:40.933896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ee81448f0f4e68956d363775bcc01c)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:40.935970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:40.995933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4aa6408e764738a2d556a33172d0b5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.093744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4aa6408e764738a2d556a33172d0b5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.094396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4aa6408e764738a2d556a33172d0b5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.097517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4aa6408e764738a2d556a33172d0b5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.111893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4aa6408e764738a2d556a33172d0b5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.143121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ee81448f0f4e68956d363775bcc01c', 'request_id': 'req-d8967942-bbb1-4a0a-936e-f66983e35804'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.188373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ee81448f0f4e68956d363775bcc01c', 'request_id': 'req-d8967942-bbb1-4a0a-936e-f66983e35804'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.189435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 824/1653] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:40 2022] DELETE /identity/v3/users/2f4aa6408e764738a2d556a33172d0b5 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:41.321238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ee81448f0f4e68956d363775bcc01c', 'request_id': 'req-d8967942-bbb1-4a0a-936e-f66983e35804'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.343920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] The token cache is being invalidate because project 78ee81448f0f4e68956d363775bcc01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:41.344135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.347086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d8967942-bbb1-4a0a-936e-f66983e35804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:41.351985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 830/1654] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:40 2022] DELETE /identity/v3/projects/78ee81448f0f4e68956d363775bcc01c => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:41.356132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:41.357626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:41.616545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78822cd-7cc6-4404-85a8-fdc75a1deb94 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:41.626904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:41.628051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:41.628456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:41.628912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] PATH_INFO: `/v3/users/4cec7ff60e9d406883219c6834e879a0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:41.631355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cec7ff60e9d406883219c6834e879a0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:41.649418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:41.668960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:41.902546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cec7ff60e9d406883219c6834e879a0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:42.019420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:42.023062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cec7ff60e9d406883219c6834e879a0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:42.033749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cec7ff60e9d406883219c6834e879a0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:42.033749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cec7ff60e9d406883219c6834e879a0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:42.050416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4172b89b-7bb0-4a07-bbab-bd9137065a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cec7ff60e9d406883219c6834e879a0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:42.125722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 825/1655] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:41 2022] DELETE /identity/v3/users/4cec7ff60e9d406883219c6834e879a0 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:42.330652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:42.332625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:42.333802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:42.439522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] MFA Rules not processed for user `feed6fb6fc4e4558ad762f9468bff195`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:42.444269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:42.578066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 831/1656] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:42.890633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2eca9-9a51-4d7c-a7e4-78ddac8fdf77 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:42.895598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2eca9-9a51-4d7c-a7e4-78ddac8fdf77 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:42.895598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2eca9-9a51-4d7c-a7e4-78ddac8fdf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:42.950656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bfc2eca9-9a51-4d7c-a7e4-78ddac8fdf77 None None] MFA Rules not processed for user `ca6855123d0740aba728a2379419cc95`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:42.954015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bfc2eca9-9a51-4d7c-a7e4-78ddac8fdf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.003692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:43.106090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:43.108147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.127934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 826/1657] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:43.145411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.145411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.145411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.212292 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] MFA Rules not processed for user `ca6855123d0740aba728a2379419cc95`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:43.216098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.237910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 827/1658] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:43.268615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:43.272228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.354508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c1689-836b-41fa-a4c2-81746cf7dfa7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:43.375096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:43.376028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.376454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.376869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] PATH_INFO: `/v3/projects/09e4028cba7c4eac9907b6e7ae29b584` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.381017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e4028cba7c4eac9907b6e7ae29b584)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:43.398930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:43.568746 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb27a6-082e-4c95-bf30-b98218dc1f81 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:43.580947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:43.586199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.586199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.586199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.605463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:43.612197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:43.638166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 828/1659] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:15:43.642920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e4028cba7c4eac9907b6e7ae29b584', 'request_id': 'req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:43.659619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e4028cba7c4eac9907b6e7ae29b584', 'request_id': 'req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:43.691215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:43.693432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.709242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:43.727270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77116a6-596a-418c-80f6-cc459813b33e service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:43.733804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e4028cba7c4eac9907b6e7ae29b584', 'request_id': 'req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:43.753935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] The token cache is being invalidate because project 09e4028cba7c4eac9907b6e7ae29b584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:43.754461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:43.755693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e22b1c71-3083-40f4-8714-ceadd3cc5ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:43.761861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 832/1660] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:43 2022] DELETE /identity/v3/projects/09e4028cba7c4eac9907b6e7ae29b584 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:15:43.765688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4064a-bda5-4eb0-bb1d-314456468fc2 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.766275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4064a-bda5-4eb0-bb1d-314456468fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.766709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4064a-bda5-4eb0-bb1d-314456468fc2 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.768576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 833/1661] 10.222.0.62 () {54 vars in 962 bytes} [Tue Sep 13 10:15:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:43.777209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabc861-08e5-49e8-ac62-2a1f5f2f94d2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.777828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabc861-08e5-49e8-ac62-2a1f5f2f94d2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.778264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabc861-08e5-49e8-ac62-2a1f5f2f94d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.810020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3cabc861-08e5-49e8-ac62-2a1f5f2f94d2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:43.812491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3cabc861-08e5-49e8-ac62-2a1f5f2f94d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.822953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:43.823814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.824311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.824801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] PATH_INFO: `/v3/projects/dd47ed6fc64a46c4a7386cb7025f9d06` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.826552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd47ed6fc64a46c4a7386cb7025f9d06)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:43.827916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 10:15:43.827916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 13 10:15:43.827916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: Sep 13 10:15:43.830977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:43.837569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 834/1662] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:43.839033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507aa6e-65c0-4eb3-8edd-1cfd75079601 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:43.839397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507aa6e-65c0-4eb3-8edd-1cfd75079601 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:43.840274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507aa6e-65c0-4eb3-8edd-1cfd75079601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:43.867760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4507aa6e-65c0-4eb3-8edd-1cfd75079601 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:43.871034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4507aa6e-65c0-4eb3-8edd-1cfd75079601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:43.991418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd47ed6fc64a46c4a7386cb7025f9d06', 'request_id': 'req-2874d562-86fb-44bd-83e5-b1ddc0a64a87'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:44.017364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 835/1663] 10.222.0.62 () {58 vars in 1059 bytes} [Tue Sep 13 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:44.025716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0e379-62fd-4941-83da-a46ce7c150b5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.026011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0e379-62fd-4941-83da-a46ce7c150b5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.027065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd47ed6fc64a46c4a7386cb7025f9d06', 'request_id': 'req-2874d562-86fb-44bd-83e5-b1ddc0a64a87'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:44.027345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0e379-62fd-4941-83da-a46ce7c150b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.076558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5cb0e379-62fd-4941-83da-a46ce7c150b5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:44.078123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5cb0e379-62fd-4941-83da-a46ce7c150b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:44.142927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd47ed6fc64a46c4a7386cb7025f9d06', 'request_id': 'req-2874d562-86fb-44bd-83e5-b1ddc0a64a87'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:44.164559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] The token cache is being invalidate because project dd47ed6fc64a46c4a7386cb7025f9d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:44.165060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:44.165785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2874d562-86fb-44bd-83e5-b1ddc0a64a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:44.177697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 829/1664] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:43 2022] DELETE /identity/v3/projects/dd47ed6fc64a46c4a7386cb7025f9d06 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:44.185299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe946-33d1-426c-bf36-1942f64b1e87 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.185604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe946-33d1-426c-bf36-1942f64b1e87 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.186602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe946-33d1-426c-bf36-1942f64b1e87 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.199042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 830/1665] 10.222.0.62 () {54 vars in 912 bytes} [Tue Sep 13 10:15:44 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:44.202987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.203561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.205454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.261462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 836/1666] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:44.264920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.265229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.265516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.272683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:44.273960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:44.296642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 831/1667] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:44.297593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:44.321203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb1f15-86f7-47d6-9d60-cedf4cfc6569 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:44.323916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:44.326720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:44.332025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:44.334058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.334342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.334612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.337422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:44.337660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:44.346516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362178c-e6ad-45fd-b3c8-a290fb8bc9b6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:44.373094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 832/1668] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:44.377597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.377899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.378180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.446877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:44.448168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:44.469808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 833/1669] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:44.475911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:44.506963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ec64b-eacc-4108-bd27-225d078da807 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:44.520812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:44.521621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 837/1670] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:44.522669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:44.528274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.528593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.528876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.529659 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:44.532057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:44.540078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d2b1-f952-4cff-af84-bc2183caf91d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:44.545772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:44.546156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.546599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.547061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.548625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:44.549253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:44.551145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:44.566085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 838/1671] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:44.567808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:44.570035 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:44.645893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa5918-4523-4f3c-8274-0810cf390e7d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:44.650399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef569f50-3fb4-4e0d-b8d7-c4f8faaa806c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:44.651200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef569f50-3fb4-4e0d-b8d7-c4f8faaa806c service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.652011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef569f50-3fb4-4e0d-b8d7-c4f8faaa806c service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.652011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef569f50-3fb4-4e0d-b8d7-c4f8faaa806c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.677763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef569f50-3fb4-4e0d-b8d7-c4f8faaa806c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:44.679464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef569f50-3fb4-4e0d-b8d7-c4f8faaa806c service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:44.717506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 834/1672] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/projects => generated 369 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:44.718470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:44.737769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 839/1673] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:44.740293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad444-7182-4600-8bbf-0d4e05cffda1 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.742562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad444-7182-4600-8bbf-0d4e05cffda1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.786850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad444-7182-4600-8bbf-0d4e05cffda1 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.786850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 840/1674] 10.222.0.62 () {54 vars in 912 bytes} [Tue Sep 13 10:15:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:44.786850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ad444-7182-4600-8bbf-0d4e05cffda1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:44.878071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:44.926368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889f98-760b-4cfb-bffd-d8b2c99271c6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:44.933672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec1f1b-7e7a-49cc-bfac-e453a98bbee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:44.934539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec1f1b-7e7a-49cc-bfac-e453a98bbee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:44.935068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec1f1b-7e7a-49cc-bfac-e453a98bbee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:44.935333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec1f1b-7e7a-49cc-bfac-e453a98bbee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:44.936116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec1f1b-7e7a-49cc-bfac-e453a98bbee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:44.938223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec1f1b-7e7a-49cc-bfac-e453a98bbee3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.030893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ad444-7182-4600-8bbf-0d4e05cffda1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.037388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.038230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.038657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.039095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.049281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.054694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 835/1675] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/projects => generated 381 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:45.056136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.056411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.060837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.065225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.173479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:45.175095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:45.223532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 841/1676] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:44 2022] POST /identity/v3/users => generated 285 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:45.224706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.239003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf442-e7f1-45a9-baae-743c9ffc7416 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.375352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 836/1677] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:15:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:45.376359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.413670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.418212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.418212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.418212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.418212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.426363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.541023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 842/1678] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:45 2022] POST /identity/v3/users => generated 291 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:45.543625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.547626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:45.559428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202e178-3c37-4a9c-accf-22a962918d67 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.579713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bb0ff-d3a4-43ee-83c3-955b9bfe7490 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.582752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.583518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.585067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.585503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.587107 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.587609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.588537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.588827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.589723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.591984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.607364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.612814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.615299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 837/1679] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:45.619847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.631891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 843/1680] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:45.633168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.634556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57312df3-2599-42ca-ab0a-8a9e3635eb2c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.645612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.646596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.647044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.647589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.654309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.654309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.657826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36771dd-37ee-4974-97ee-452f6fd78f1b service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.673375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.674013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.674297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.674575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.675321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.680982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.685024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 838/1681] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:45.686906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.707561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329cee8-9118-4954-8e80-0dd860dcef4a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.708635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 844/1682] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:45.709524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.721224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.721867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.722156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.722490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.723681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.725436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.733307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feca42-f621-4b53-ae0a-bae475c861a1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.744553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385f1a-2a2e-4c11-9bf4-0327357c504f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.745256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385f1a-2a2e-4c11-9bf4-0327357c504f admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.745548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385f1a-2a2e-4c11-9bf4-0327357c504f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.745828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385f1a-2a2e-4c11-9bf4-0327357c504f admin admin] PATH_INFO: `/v3/projects/fa55cd5c1ad1428986f7109797dd979d/users/29d27bc889c648d78575e24c294d5f08/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.750177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:45.755288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 839/1683] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:45.758541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:45.781187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d318-3cc4-4ef4-b68d-1c7ec21d8560 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:45.789887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bc9f1-4a2b-4b2b-b73e-57d7f93a803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:45.790931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc9f1-4a2b-4b2b-b73e-57d7f93a803d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.791427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc9f1-4a2b-4b2b-b73e-57d7f93a803d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.791982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385f1a-2a2e-4c11-9bf4-0327357c504f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa55cd5c1ad1428986f7109797dd979d, user_id=29d27bc889c648d78575e24c294d5f08, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.792370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc9f1-4a2b-4b2b-b73e-57d7f93a803d admin admin] PATH_INFO: `/v3/projects/fd5b0cd2ca6247ee8d45c28f1c01e0d5/users/429336a1ae734517ad86bd576e9e24c1/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:45.795485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385f1a-2a2e-4c11-9bf4-0327357c504f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.809757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bc9f1-4a2b-4b2b-b73e-57d7f93a803d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5b0cd2ca6247ee8d45c28f1c01e0d5, user_id=429336a1ae734517ad86bd576e9e24c1, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:45.813367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bc9f1-4a2b-4b2b-b73e-57d7f93a803d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:45.934367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 845/1684] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:45 2022] PUT /identity/v3/projects/fa55cd5c1ad1428986f7109797dd979d/users/29d27bc889c648d78575e24c294d5f08/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:45.944610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:45.945271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:45.945769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:46.002521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 840/1685] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:45 2022] PUT /identity/v3/projects/fd5b0cd2ca6247ee8d45c28f1c01e0d5/users/429336a1ae734517ad86bd576e9e24c1/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:46.016328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:46.016629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:46.016912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:46.023737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] MFA Rules not processed for user `29d27bc889c648d78575e24c294d5f08`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:46.025860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:46.090757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] MFA Rules not processed for user `429336a1ae734517ad86bd576e9e24c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:46.096651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:46.168121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 846/1686] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:46.171921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:46.255496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 841/1687] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:15:46.256499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:46.303091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed06d-c69b-47ed-8e80-013c4be64822 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:46.309719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:46.311351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:46.311797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:46.313416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:46.314318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:46.317295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:46.393268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed6e19-4e5e-4b73-b3a9-489e715a621e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:46.406229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:46.407737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:46.409430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:46.409826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:46.411790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:46.417006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:46.459433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 847/1688] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:46 2022] POST /identity/v3/projects => generated 367 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:46.463589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:46.551264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 842/1689] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:46 2022] POST /identity/v3/projects => generated 379 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:46.555829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:46.665073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749573b0-13fa-4b95-8df5-d4cc8d880766 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:46.733637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22de9-9afa-45b4-bcf6-23c49e4f5058 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:46.754970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:46.760174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:46.760574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:46.761962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:46.761962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:46.762773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:46.763517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:46.827983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:46.828632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:46.828918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:46.829520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:46.829835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:46.836558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:46.887921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 843/1690] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:46 2022] POST /identity/v3/users => generated 296 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:46.888588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:46.900426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:46.993172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 848/1691] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:46 2022] POST /identity/v3/users => generated 290 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:46.993172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.018297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90ff5-e341-4726-aefa-7312921429b8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.039678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95abb-8c01-41da-b91c-051a9d16a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.040318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95abb-8c01-41da-b91c-051a9d16a917 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.040609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95abb-8c01-41da-b91c-051a9d16a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.040889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95abb-8c01-41da-b91c-051a9d16a917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.041552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95abb-8c01-41da-b91c-051a9d16a917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.043284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95abb-8c01-41da-b91c-051a9d16a917 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.083879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbead5-662a-4168-aac1-e6d3c604a4f4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.091526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 849/1692] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:47.096264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88af6a-2da3-4b93-ae6e-bab8139001c3 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.096563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88af6a-2da3-4b93-ae6e-bab8139001c3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.096845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88af6a-2da3-4b93-ae6e-bab8139001c3 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.099714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 850/1693] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:47.099948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88af6a-2da3-4b93-ae6e-bab8139001c3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.107553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.108434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.108960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.109388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.114053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:47.118131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88af6a-2da3-4b93-ae6e-bab8139001c3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.124561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.125238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.125527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.125849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.126581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.128493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.146029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 851/1694] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:47.146029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.157325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff52e7e-cdcf-4cdf-8209-de31c810125b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.157873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.162203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.167996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f707f-e910-41a0-9a95-d48fb0c336e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.168960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f707f-e910-41a0-9a95-d48fb0c336e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.169245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f707f-e910-41a0-9a95-d48fb0c336e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.169521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f707f-e910-41a0-9a95-d48fb0c336e4 admin admin] PATH_INFO: `/v3/projects/36228d16965441a6a77f35d1a70cbd26/users/a9f6ec3fe6064b7a8d0bc10b37fbd1d0/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.206336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 844/1695] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:47.211854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.212102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f707f-e910-41a0-9a95-d48fb0c336e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36228d16965441a6a77f35d1a70cbd26, user_id=a9f6ec3fe6064b7a8d0bc10b37fbd1d0, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.216890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f707f-e910-41a0-9a95-d48fb0c336e4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.276018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4cf32-eaee-47ec-ab77-371933029f99 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.287359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.288170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.288453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.288750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] PATH_INFO: `/v3/projects/d34f1a6845954647b4830afc3193f730/users/af64ad4cc26b4d7293410a7877ea0d07/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.335436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34f1a6845954647b4830afc3193f730, user_id=af64ad4cc26b4d7293410a7877ea0d07, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.338273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.344668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 852/1696] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:47 2022] PUT /identity/v3/projects/36228d16965441a6a77f35d1a70cbd26/users/a9f6ec3fe6064b7a8d0bc10b37fbd1d0/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:47.346307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.346602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.346880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.430899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:47.441405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:47.442711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:47.456314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 845/1697] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:47 2022] PUT /identity/v3/projects/d34f1a6845954647b4830afc3193f730/users/af64ad4cc26b4d7293410a7877ea0d07/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:47.458196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.668209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 853/1698] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:47.671453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.729321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2c430-0fe5-41ec-9b02-5b016e11b80a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.744709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.745767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.746471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.746914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.747826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.749674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.774488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 846/1699] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:47.778641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:47.814300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3be2-5877-40b1-80ac-aaa37caa298f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.828012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749069a-efd0-45cf-8287-a2273fc560a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:47.829387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749069a-efd0-45cf-8287-a2273fc560a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.829914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749069a-efd0-45cf-8287-a2273fc560a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.831534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749069a-efd0-45cf-8287-a2273fc560a4 admin admin] PATH_INFO: `/v3/projects/36228d16965441a6a77f35d1a70cbd26/users/a9f6ec3fe6064b7a8d0bc10b37fbd1d0/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.836207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749069a-efd0-45cf-8287-a2273fc560a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36228d16965441a6a77f35d1a70cbd26, user_id=a9f6ec3fe6064b7a8d0bc10b37fbd1d0, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:47.841944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:47.843912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749069a-efd0-45cf-8287-a2273fc560a4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:47.874713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 847/1700] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:47 2022] PUT /identity/v3/projects/36228d16965441a6a77f35d1a70cbd26/users/a9f6ec3fe6064b7a8d0bc10b37fbd1d0/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:47.877702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2674a6-7477-4e39-a6f8-f1ef659ef102 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.877998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2674a6-7477-4e39-a6f8-f1ef659ef102 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.878279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2674a6-7477-4e39-a6f8-f1ef659ef102 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.883547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 848/1701] 10.222.0.62 () {54 vars in 961 bytes} [Tue Sep 13 10:15:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:47.889234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:47.889696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:47.890091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:47.896426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c344e-465b-43f5-a8bf-1c197d2c39df None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:47.975261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] MFA Rules not processed for user `a9f6ec3fe6064b7a8d0bc10b37fbd1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:47.979091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:48.058452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819a721-043f-4bb7-a2f2-b79ba4ac2434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:48.060651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819a721-043f-4bb7-a2f2-b79ba4ac2434 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.060651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819a721-043f-4bb7-a2f2-b79ba4ac2434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.060651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819a721-043f-4bb7-a2f2-b79ba4ac2434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.060651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819a721-043f-4bb7-a2f2-b79ba4ac2434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:48.062333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819a721-043f-4bb7-a2f2-b79ba4ac2434 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:48.087489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 854/1702] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:48.087489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cd5b6-88cd-41b8-bec7-b749911e6f11 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.087489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cd5b6-88cd-41b8-bec7-b749911e6f11 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.087489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cd5b6-88cd-41b8-bec7-b749911e6f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.123153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 849/1703] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:15:48.124054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:48.144376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c77cd5b6-88cd-41b8-bec7-b749911e6f11 None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:48.145718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c77cd5b6-88cd-41b8-bec7-b749911e6f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:48.156777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e62e1-7b63-4436-a75d-35193e37b74b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:48.167020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:48.170766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.170766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.170766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] PATH_INFO: `/v3/projects/d34f1a6845954647b4830afc3193f730/users/af64ad4cc26b4d7293410a7877ea0d07/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.178270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34f1a6845954647b4830afc3193f730, user_id=af64ad4cc26b4d7293410a7877ea0d07, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:48.186030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:48.231157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 850/1704] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:48 2022] PUT /identity/v3/projects/d34f1a6845954647b4830afc3193f730/users/af64ad4cc26b4d7293410a7877ea0d07/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:48.231157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:48.232151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:48.328743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 855/1705] 10.222.0.62 () {58 vars in 1058 bytes} [Tue Sep 13 10:15:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:48.330501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.331432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.333701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.364380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c5480-a743-42bb-aa0b-09ddccf09bf6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:48.370705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:48.371625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.371967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.372224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.416261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] MFA Rules not processed for user `af64ad4cc26b4d7293410a7877ea0d07`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:48.418001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:48.596591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 856/1706] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:48.598072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:48.608554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:48.612139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:48.655385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 851/1707] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:48.683201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:48.731066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fda62-76a0-4a03-87e2-b39939f93b69 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:48.750269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:48.750791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.751077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.751371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.775858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:48.787536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:48.789971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536f9ba-11f8-42c6-8b7d-9178cd2e08b9 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:48.795507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:48.800083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16101acf-56a6-4fb0-81a4-a7cf4adb15ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:48.801703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16101acf-56a6-4fb0-81a4-a7cf4adb15ca service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:48.802170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16101acf-56a6-4fb0-81a4-a7cf4adb15ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:48.803147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16101acf-56a6-4fb0-81a4-a7cf4adb15ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:48.820800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 857/1708] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:48.878858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16101acf-56a6-4fb0-81a4-a7cf4adb15ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:48.880248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16101acf-56a6-4fb0-81a4-a7cf4adb15ca service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:48.938464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 852/1709] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:49.307764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:49.328145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83e3d3-810d-44bf-8134-0bc6b04e2713 service placement] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:49.337379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:49.338058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:49.338341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:49.338614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:49.494467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:49.494955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:49.495029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:49.516103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:49.518346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:49.536283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 858/1710] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:49.559368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:49.561876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] MFA Rules not processed for user `c069999293a847cdb11650045e22576c`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:49.571800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:49.693594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 853/1711] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:15:49.709257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:49.720205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:49.858145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969c2c8-aea2-4aa2-b3a8-ca754e71402d service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:49.880665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:49.881127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:49.881423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:49.882023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:49.932933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:49.944956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:49.970692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 859/1712] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:50.770678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:50.773298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:50.852231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:50.962905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e610cb-02b2-4366-b60b-1d2dd5e1a915 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:50.983486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:50.983486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:50.983486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:50.983486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] PATH_INFO: `/v3/users/af64f2a8387e4996bd91f9aadfd792d4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:50.983917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=af64f2a8387e4996bd91f9aadfd792d4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:50.985273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:51.349970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af64f2a8387e4996bd91f9aadfd792d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:51.458282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af64f2a8387e4996bd91f9aadfd792d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:51.460354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af64f2a8387e4996bd91f9aadfd792d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:51.460354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af64f2a8387e4996bd91f9aadfd792d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:51.474872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af64f2a8387e4996bd91f9aadfd792d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:51.527054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 854/1713] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:50 2022] DELETE /identity/v3/users/af64f2a8387e4996bd91f9aadfd792d4 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:51.825415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:52.293304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:52.329048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbab9-2733-4bbb-aa14-21964f4bc971 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:52.470432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:52.470432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:52.470432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:52.470432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] PATH_INFO: `/v3/projects/bab9fc25a2364ae79578ccc87de8bc35` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:52.476521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab9fc25a2364ae79578ccc87de8bc35)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:52.478495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:52.707261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab9fc25a2364ae79578ccc87de8bc35', 'request_id': 'req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:52.722964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab9fc25a2364ae79578ccc87de8bc35', 'request_id': 'req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:52.820944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab9fc25a2364ae79578ccc87de8bc35', 'request_id': 'req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:52.871693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] The token cache is being invalidate because project bab9fc25a2364ae79578ccc87de8bc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:52.871693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:52.872575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2c3c9e0-91ba-4e2e-b523-4f2091ca42b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:52.879394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 860/1714] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:52 2022] DELETE /identity/v3/projects/bab9fc25a2364ae79578ccc87de8bc35 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:52.882944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:52.884375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:52.903391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427528-c320-4ac3-9f5b-4556837bb7bf admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:53.025254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:53.042518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:53.043349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:53.043710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:53.044860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] PATH_INFO: `/v3/users/feed6fb6fc4e4558ad762f9468bff195` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:53.048062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feed6fb6fc4e4558ad762f9468bff195)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:53.056544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:53.299988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c497e2-48d1-45b4-8c49-24958500833a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:53.301478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c497e2-48d1-45b4-8c49-24958500833a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:53.301952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c497e2-48d1-45b4-8c49-24958500833a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:53.311318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feed6fb6fc4e4558ad762f9468bff195'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:53.405453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-94c497e2-48d1-45b4-8c49-24958500833a None None] MFA Rules not processed for user `1cc3e06e1b1440c18c0fb902beeed27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:53.407110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-94c497e2-48d1-45b4-8c49-24958500833a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:53.453085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feed6fb6fc4e4558ad762f9468bff195'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:53.453709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feed6fb6fc4e4558ad762f9468bff195'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:53.454358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feed6fb6fc4e4558ad762f9468bff195'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:53.468096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e6d8ca02-51bb-4b39-9c42-48542fc6a059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feed6fb6fc4e4558ad762f9468bff195'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:53.524898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 855/1715] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:52 2022] DELETE /identity/v3/users/feed6fb6fc4e4558ad762f9468bff195 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:53.532865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:53.535079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:53.535576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:53.556487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 861/1716] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:15:53.571067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:53.572172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:53.572857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:53.615267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] MFA Rules not processed for user `1cc3e06e1b1440c18c0fb902beeed27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:53.617612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:53.676180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:53.691833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:53.799130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 862/1717] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:53.809963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:53.816544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:53.872555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 856/1718] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:53.996875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:54.004209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7fa60-9a35-4058-97b7-f86c856b58d6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:54.012220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc54d2-d395-4d21-96ce-dff3be8388d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:54.012919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc54d2-d395-4d21-96ce-dff3be8388d3 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:54.013266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc54d2-d395-4d21-96ce-dff3be8388d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:54.013550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc54d2-d395-4d21-96ce-dff3be8388d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:54.032443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc54d2-d395-4d21-96ce-dff3be8388d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:54.033867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc54d2-d395-4d21-96ce-dff3be8388d3 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:54.047373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 863/1719] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:54.086983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:54.389633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:54.391445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:54.416052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:54.416959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:54.417438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:54.585636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] MFA Rules not processed for user `4067d48268f84854af4d672ab3f9dc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:54.589302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:54.720739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:54.797112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82589294-4d11-457c-865e-03a0a232fdcd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:54.813355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:54.814840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:54.819874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:54.820330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:54.838342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:54.840595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:54.877449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 857/1720] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:15:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:15:54.877449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:54.877729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 864/1721] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:54.877729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:54.877971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:54.880688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:55.187643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaade25-339a-439d-9bc2-56c6454a3f8f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:55.193556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:55.200165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169237d1-9eb5-40ae-a443-550f491a9b97 service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:55.204827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:55.205553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:55.207843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:55.208211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] PATH_INFO: `/v3/users/9e68f00088ac4ec4a145eb4d39a8b97f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:55.209951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:55.211229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:55.211850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e68f00088ac4ec4a145eb4d39a8b97f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:55.214905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:55.218035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:55.241929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] PATH_INFO: `/v3/projects/cb6c730fed914bb3906059b9b58b1195` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:55.241929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6c730fed914bb3906059b9b58b1195)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:55.241929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:55.580628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e68f00088ac4ec4a145eb4d39a8b97f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.581243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6c730fed914bb3906059b9b58b1195', 'request_id': 'req-be6b5d90-7e77-44ee-aa62-363831f919f5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.591449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6c730fed914bb3906059b9b58b1195', 'request_id': 'req-be6b5d90-7e77-44ee-aa62-363831f919f5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.759474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e68f00088ac4ec4a145eb4d39a8b97f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.764372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e68f00088ac4ec4a145eb4d39a8b97f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.764372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e68f00088ac4ec4a145eb4d39a8b97f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.772971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e68f00088ac4ec4a145eb4d39a8b97f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.795750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 865/1722] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:54 2022] DELETE /identity/v3/users/9e68f00088ac4ec4a145eb4d39a8b97f => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:55.798676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:55.804984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:55.847618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6c730fed914bb3906059b9b58b1195', 'request_id': 'req-be6b5d90-7e77-44ee-aa62-363831f919f5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.873041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] The token cache is being invalidate because project cb6c730fed914bb3906059b9b58b1195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:55.873352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.880902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be6b5d90-7e77-44ee-aa62-363831f919f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:55.890089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 858/1723] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:54 2022] DELETE /identity/v3/projects/cb6c730fed914bb3906059b9b58b1195 => generated 0 bytes in 1020 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:55.896715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddcc21-2d96-4ece-a3d1-694cd935685c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:55.898379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddcc21-2d96-4ece-a3d1-694cd935685c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:55.898688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddcc21-2d96-4ece-a3d1-694cd935685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:55.951011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-14ddcc21-2d96-4ece-a3d1-694cd935685c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:55.952391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-14ddcc21-2d96-4ece-a3d1-694cd935685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:56.065060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fd57b-2876-44cc-a0d7-5dc4b2d823e7 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:56.096730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:56.102263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.103215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.103651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.109570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:56.163073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 859/1724] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:56.163983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.164498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.165103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.212822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:56.214082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:56.219803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:56.223808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:56.234536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 860/1725] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:56.235395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:56.239867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 866/1726] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:56.241397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:56.242781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:56.244933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:56.262932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4225947-e89b-4a20-8d02-7d3bb995baaa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:56.276011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:56.276737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.276928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.277198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] PATH_INFO: `/v3/users/29d27bc889c648d78575e24c294d5f08` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.278370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d27bc889c648d78575e24c294d5f08)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:56.280586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:56.471833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854912-6787-4f6e-b1a0-d7aa035d3292 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:56.488762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939fdfa-ec44-43b1-8dad-5d8c7f1e0df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:56.498188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939fdfa-ec44-43b1-8dad-5d8c7f1e0df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.500436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939fdfa-ec44-43b1-8dad-5d8c7f1e0df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.501216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939fdfa-ec44-43b1-8dad-5d8c7f1e0df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.530851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939fdfa-ec44-43b1-8dad-5d8c7f1e0df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:56.535905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939fdfa-ec44-43b1-8dad-5d8c7f1e0df0 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:56.561340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 867/1727] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:15:56.564569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.564569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.566254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.616308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:56.618338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d27bc889c648d78575e24c294d5f08'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:56.623525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:56.658983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 868/1728] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:56.667226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:56.701497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cc242-c0d6-4c4b-85c9-18539e165985 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:56.723572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:56.724359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.724833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.725139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.725840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:56.726152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:56.728780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:56.749060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 869/1729] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:56.753260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:56.776146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ce003-1df3-4078-9978-9838d0811f0e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:56.802935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d27bc889c648d78575e24c294d5f08'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:56.808277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d27bc889c648d78575e24c294d5f08'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:56.808412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:56.808412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.808711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d27bc889c648d78575e24c294d5f08'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:56.808884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.808884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.810383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:56.815933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:56.827603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8859b3b8-de39-4125-bf3e-3674c9232027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d27bc889c648d78575e24c294d5f08'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:56.873429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 861/1730] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:56 2022] DELETE /identity/v3/users/29d27bc889c648d78575e24c294d5f08 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:56.879740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:56.880520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:56.880696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:56.946521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:56.950246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:57.013888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 870/1731] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:56 2022] POST /identity/v3/projects => generated 365 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:57.015197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:57.020496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:57.126100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:57.235650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 862/1732] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:57.272048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:57.272048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:57.322551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa18c2-2d12-45b4-80dd-7e8d1bdad79f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:57.339704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6071-f543-4874-8a3c-d8d87f3940d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:57.340466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6071-f543-4874-8a3c-d8d87f3940d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:57.340934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6071-f543-4874-8a3c-d8d87f3940d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:57.341327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6071-f543-4874-8a3c-d8d87f3940d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:57.342079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf6071-f543-4874-8a3c-d8d87f3940d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:57.344350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf6071-f543-4874-8a3c-d8d87f3940d7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:57.468096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86625a-fbab-4977-8560-5aa436b4fa02 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:57.491880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 871/1733] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:57 2022] POST /identity/v3/users => generated 283 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:57.491880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:57.491880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:57.493159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:57.560258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] MFA Rules not processed for user `1cc3e06e1b1440c18c0fb902beeed27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:57.563474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:57.661176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:57.666250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:57.666699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:57.667085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:57.747382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:57.747382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:57.750744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 872/1734] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:57.754380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:57.759562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 863/1735] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:15:57.760765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:57.763907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:57.773017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ec000-6d36-4513-990b-8ceb9a3f5697 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:57.789257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23876835-773a-479b-80fd-8b242a4b03ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:57.790148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876835-773a-479b-80fd-8b242a4b03ca admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:57.790723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876835-773a-479b-80fd-8b242a4b03ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:57.791149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876835-773a-479b-80fd-8b242a4b03ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:57.791992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23876835-773a-479b-80fd-8b242a4b03ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:57.793827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23876835-773a-479b-80fd-8b242a4b03ca admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:57.798466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e181c-d739-4643-a473-77af3a906c90 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:57.815104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 873/1736] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:57.815104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:57.815104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:57.815104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:57.839542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] MFA Rules not processed for user `c069999293a847cdb11650045e22576c`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:57.839651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:57.839651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:57.839830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:57.839928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:57.841138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] PATH_INFO: `/v3/projects/87d1f004835544b6bbbfe6f90231187b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:57.843204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d1f004835544b6bbbfe6f90231187b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:57.849071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:57.944638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 874/1737] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:57.946743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:58.002881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6e78-de5d-42cc-9c85-65482f04766d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:58.013398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:58.014225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.014600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.015257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.016500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:58.018443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:58.046000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 875/1738] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:58.047185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:58.071009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64c81-48ce-47f7-b7ac-ff493c71cc14 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:58.075878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:58.098277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d1f004835544b6bbbfe6f90231187b', 'request_id': 'req-8567de7c-9aa0-4918-90de-076d717dfa5e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:58.108259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d1f004835544b6bbbfe6f90231187b', 'request_id': 'req-8567de7c-9aa0-4918-90de-076d717dfa5e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:58.176708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:15:58.204436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544c5ed-2fb6-40b7-86b7-88e0f2b7898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:58.204570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d1f004835544b6bbbfe6f90231187b', 'request_id': 'req-8567de7c-9aa0-4918-90de-076d717dfa5e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:58.208229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544c5ed-2fb6-40b7-86b7-88e0f2b7898f admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.210568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544c5ed-2fb6-40b7-86b7-88e0f2b7898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.210568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544c5ed-2fb6-40b7-86b7-88e0f2b7898f admin admin] PATH_INFO: `/v3/projects/c55232d1d89247378fac6a2d591c2930/users/5bda445222c44eb09b215e27fe40a0f8/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.247958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] The token cache is being invalidate because project 87d1f004835544b6bbbfe6f90231187b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:58.248533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:58.250528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544c5ed-2fb6-40b7-86b7-88e0f2b7898f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55232d1d89247378fac6a2d591c2930, user_id=5bda445222c44eb09b215e27fe40a0f8, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:58.253844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8567de7c-9aa0-4918-90de-076d717dfa5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:58.255740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544c5ed-2fb6-40b7-86b7-88e0f2b7898f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:58.260086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 864/1739] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:57 2022] DELETE /identity/v3/projects/87d1f004835544b6bbbfe6f90231187b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:58.277911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfef09-1317-45a4-be50-5e3d9f33da8f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.278370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfef09-1317-45a4-be50-5e3d9f33da8f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.278538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfef09-1317-45a4-be50-5e3d9f33da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.350423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-afbfef09-1317-45a4-be50-5e3d9f33da8f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:58.351710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-afbfef09-1317-45a4-be50-5e3d9f33da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:58.355229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 876/1740] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:15:58 2022] PUT /identity/v3/projects/c55232d1d89247378fac6a2d591c2930/users/5bda445222c44eb09b215e27fe40a0f8/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:58.367077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.367390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.367716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.459752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] MFA Rules not processed for user `5bda445222c44eb09b215e27fe40a0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:58.461001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:58.551434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 865/1741] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:58.559977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.560462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.560882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.609886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:58.612279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:58.637328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 866/1742] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:15:58.642177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:58.668168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf291263-2821-498b-b35d-83538c3ef44f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:58.670178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 877/1743] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:58.675991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:58.677515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:58.685729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:58.685729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.685925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.686060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.687815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:15:58.691438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:58.693086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:58.707021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544a746-1e47-4b9b-97ff-58e73d3fcdbe None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:58.713964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:58.714876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.714992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.715361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.716201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:58.720778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 867/1744] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:15:58.720944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:58.723150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:58.765303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fef509-8939-47ff-b800-fba8fb9ebc46 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:58.783826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:58.789097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:58.789097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:58.789097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:58.790640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:58.805727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:58.853231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 878/1745] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:58 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:58.854249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:58.858038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:58.966617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01476504-97bc-47c7-bdaf-1ee3405268ac admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:58.970045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 868/1746] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:15:58 2022] POST /identity/v3/projects => generated 387 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:58.972501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:59.058612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:59.058612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.058612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.058612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] PATH_INFO: `/v3/projects/fa55cd5c1ad1428986f7109797dd979d` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.060150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa55cd5c1ad1428986f7109797dd979d)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:59.062576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:59.115413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3f5f-6c30-4159-afce-e0d92eec5dd8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:59.202428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5202364-8469-4e93-a645-f297bededb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:59.204890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5202364-8469-4e93-a645-f297bededb01 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.204890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5202364-8469-4e93-a645-f297bededb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.204890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5202364-8469-4e93-a645-f297bededb01 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.205212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5202364-8469-4e93-a645-f297bededb01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:59.209698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5202364-8469-4e93-a645-f297bededb01 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:59.410006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 869/1747] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:58 2022] POST /identity/v3/users => generated 289 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:59.414628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a62ca7-bd75-471c-b5f0-4cdc99f32cf7 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.421619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a62ca7-bd75-471c-b5f0-4cdc99f32cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.423660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a62ca7-bd75-471c-b5f0-4cdc99f32cf7 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.430205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 870/1748] 10.222.0.62 () {54 vars in 912 bytes} [Tue Sep 13 10:15:59 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:15:59.434190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a62ca7-bd75-471c-b5f0-4cdc99f32cf7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:59.454163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa55cd5c1ad1428986f7109797dd979d', 'request_id': 'req-dc2f0682-cde9-4ec4-8e32-db729f46a138'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:59.479940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa55cd5c1ad1428986f7109797dd979d', 'request_id': 'req-dc2f0682-cde9-4ec4-8e32-db729f46a138'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:59.562450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa55cd5c1ad1428986f7109797dd979d', 'request_id': 'req-dc2f0682-cde9-4ec4-8e32-db729f46a138'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:59.585114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] The token cache is being invalidate because project fa55cd5c1ad1428986f7109797dd979d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:15:59.585782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:59.601396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dc2f0682-cde9-4ec4-8e32-db729f46a138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:15:59.605963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 879/1749] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:15:58 2022] DELETE /identity/v3/projects/fa55cd5c1ad1428986f7109797dd979d => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:15:59.613311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.615797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.617717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.646476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:15:59.648536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:59.704432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a62ca7-bd75-471c-b5f0-4cdc99f32cf7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:59.720575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fcebd-3da5-4da3-8a45-06713fe31ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:59.723879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcebd-3da5-4da3-8a45-06713fe31ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.725611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcebd-3da5-4da3-8a45-06713fe31ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.726057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcebd-3da5-4da3-8a45-06713fe31ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.735156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fcebd-3da5-4da3-8a45-06713fe31ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:59.737313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fcebd-3da5-4da3-8a45-06713fe31ab1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:59.805426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 880/1750] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Sep 13 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:15:59.806971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:59.809293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:15:59.862217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8986-1ee3-4383-b3d3-0142ea133ed3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:15:59.893304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:15:59.895126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.895126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.895126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.895457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:15:59.903699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:15:59.973051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 871/1751] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:15:59 2022] POST /identity/v3/users => generated 294 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:15:59.974833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:15:59.975372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:15:59.976006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:15:59.982006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 881/1752] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:15:59.984226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:15:59.995870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.017601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:00.019644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.039657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 872/1753] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:00.039657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:00.040595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.074095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008578b-9d12-4b73-8026-7e711ee721a1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:00.075776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eebfd6-2cd4-4191-826a-8822133f2a87 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:00.082465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:00.083218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:00.083503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:00.083817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:00.085917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:00.086587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:00.086875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:00.087153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] PATH_INFO: `/v3/users/af64ad4cc26b4d7293410a7877ea0d07` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:00.091873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] RBAC: Authorizing `identity:delete_user(user_id=af64ad4cc26b4d7293410a7877ea0d07)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:00.094027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:00.109295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:00.112967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:00.129118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 873/1754] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:00.129118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:00.191591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b3ad-b1ff-44a8-afb1-78815d2d2c86 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:00.204905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:00.207295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:00.208369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:00.209675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] PATH_INFO: `/v3/projects/9bb601e9f14e440dae0b8cd2df8ba22c/users/f6edda3e45a54a7f9ec9bce0d311636a/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:00.238852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb601e9f14e440dae0b8cd2df8ba22c, user_id=f6edda3e45a54a7f9ec9bce0d311636a, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:00.241973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:00.361206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 874/1755] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:00 2022] PUT /identity/v3/projects/9bb601e9f14e440dae0b8cd2df8ba22c/users/f6edda3e45a54a7f9ec9bce0d311636a/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:00.361206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:00.368674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.449907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af64ad4cc26b4d7293410a7877ea0d07'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:00.532834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9b4de-4163-4585-9e89-d2709d973a3b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:00.542012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:00.542711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:00.543073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:00.545210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:00.547672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:00.550843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:00.576017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 875/1756] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:00.576257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af64ad4cc26b4d7293410a7877ea0d07'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:00.577130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af64ad4cc26b4d7293410a7877ea0d07'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:00.577754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:00.577887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af64ad4cc26b4d7293410a7877ea0d07'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:00.588771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.592622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af64ad4cc26b4d7293410a7877ea0d07'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:00.699810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 882/1757] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:15:59 2022] DELETE /identity/v3/users/af64ad4cc26b4d7293410a7877ea0d07 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:00.703319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:00.704604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.720445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9bcd2-5cd2-40c2-a623-73268c53b7bd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:00.917013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:00.919566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:00.919860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:00.920159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:00.925180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:00.930184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea394de-7e07-432e-afcc-5d39e59938da admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:00.941333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:00.942033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:00.942494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:00.943005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:00.943836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:00.945730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:00.971134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:00.972686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 883/1758] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:00.973207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:00.973982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:00.979342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:01.020177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220963e-358c-4ce1-939e-de4ebf8f268f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:01.032819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 876/1759] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:01.034412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:01.036019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:01.038030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:01.038530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:01.038956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:01.040001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:01.041964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:01.062914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdb98-675d-40be-a250-8ff699727092 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:01.070074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9481679-2608-4fdd-b242-99c8a40277d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:01.070985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9481679-2608-4fdd-b242-99c8a40277d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:01.071427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9481679-2608-4fdd-b242-99c8a40277d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:01.078882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9481679-2608-4fdd-b242-99c8a40277d6 admin admin] PATH_INFO: `/v3/projects/9bb601e9f14e440dae0b8cd2df8ba22c/users/f6edda3e45a54a7f9ec9bce0d311636a/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:01.078882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9481679-2608-4fdd-b242-99c8a40277d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb601e9f14e440dae0b8cd2df8ba22c, user_id=f6edda3e45a54a7f9ec9bce0d311636a, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:01.079283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 884/1760] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:01.079381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9481679-2608-4fdd-b242-99c8a40277d6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:01.082722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:01.118373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 877/1761] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:01 2022] PUT /identity/v3/projects/9bb601e9f14e440dae0b8cd2df8ba22c/users/f6edda3e45a54a7f9ec9bce0d311636a/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:01.127698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657c95d-3d26-4cae-aefb-68e3f2c3742f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:01.132862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:01.134464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:01.134854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:01.243268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] MFA Rules not processed for user `f6edda3e45a54a7f9ec9bce0d311636a`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:01.252407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:01.320475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d4d12-549c-4667-b63e-90f758ab1b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:01.320888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d4d12-549c-4667-b63e-90f758ab1b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:01.321182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d4d12-549c-4667-b63e-90f758ab1b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:01.325862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d4d12-549c-4667-b63e-90f758ab1b8a admin admin] PATH_INFO: `/v3/projects/c89cf0f3b72f47b391dcef94d15fb133/users/1a75206ab6a6432f9687a2deedc38d14/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:01.390427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d4d12-549c-4667-b63e-90f758ab1b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89cf0f3b72f47b391dcef94d15fb133, user_id=1a75206ab6a6432f9687a2deedc38d14, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:01.399704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d4d12-549c-4667-b63e-90f758ab1b8a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:01.457419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 878/1762] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:01.471417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:01.474224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:01.544717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 885/1763] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:01 2022] PUT /identity/v3/projects/c89cf0f3b72f47b391dcef94d15fb133/users/1a75206ab6a6432f9687a2deedc38d14/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:01.561317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:01.561639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:01.561951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:01.595450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4814f5-25da-4152-8996-ab1d627c02ed None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:01.625311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] MFA Rules not processed for user `1a75206ab6a6432f9687a2deedc38d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:01.649399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:01.776331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:01.778638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:01.779840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:01.783880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:01.784377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:01.851316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:01.892772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 886/1764] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:01.897676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:01.971534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:01.980323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:02.013371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 879/1765] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:02.244706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69d409-3e08-4fcf-92aa-ee8c40e32b93 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:02.269543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:02.283064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:02.283402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:02.287362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:02.291774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:02.297970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:02.514201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 887/1766] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 614 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:02.515631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:02.526286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:02.541295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496dd38-a48d-49c3-8028-2178dd176a4c service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:02.754637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:02.756012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:02.756583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:02.757003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:02.758596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:02.762023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:02.863153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee105-3eeb-4c9e-b8fd-103d9e50279f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:02.884011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:02.884011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:02.884011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:02.884011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:02.887369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:02.891194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 880/1767] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:02 2022] POST /identity/v3/users => generated 299 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:02.898267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:02.957661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37df14-78e1-4254-9964-f4c4b6fc4b47 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:02.977680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:02.980176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:02.981249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:02.982032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:02.984634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:02.991071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:03.035112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 881/1768] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:03.036573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:03.069325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08ee16-1683-4163-a190-c7a139ebf4b7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:03.085304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363972fe-6ecf-4068-849d-13908358533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:03.086775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363972fe-6ecf-4068-849d-13908358533c admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.089112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363972fe-6ecf-4068-849d-13908358533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.090120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363972fe-6ecf-4068-849d-13908358533c admin admin] PATH_INFO: `/v3/projects/012b24c2fd114e8aa9f8a8fa981c1543/users/7faa29a099c04e2b83e5e41003f6f9a6/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.097300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:03.097875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a523-ec51-4295-9115-f378b9f759c6 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:03.121529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363972fe-6ecf-4068-849d-13908358533c admin admin] RBAC: Authorizing `identity:create_grant(project_id=012b24c2fd114e8aa9f8a8fa981c1543, user_id=7faa29a099c04e2b83e5e41003f6f9a6, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:03.126607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363972fe-6ecf-4068-849d-13908358533c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:03.130215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 888/1769] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:03.155810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3705a3-651a-478d-876c-c525298a9bd5 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.156101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3705a3-651a-478d-876c-c525298a9bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.156370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3705a3-651a-478d-876c-c525298a9bd5 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.176405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 889/1770] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:16:03 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:16:03.231908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 882/1771] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:03 2022] PUT /identity/v3/projects/012b24c2fd114e8aa9f8a8fa981c1543/users/7faa29a099c04e2b83e5e41003f6f9a6/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:03.232007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3705a3-651a-478d-876c-c525298a9bd5 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:03.249650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.249939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.250269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.305994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:03.317128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:03.535028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 883/1772] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:16:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:03.537807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:03.543991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:03.564394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3705a3-651a-478d-876c-c525298a9bd5 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:03.573193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:03.574064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.575145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.575571 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.577970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:03.582973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:03.601884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 890/1773] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:03.603921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf1c0-9901-48e4-8e3f-1ad4ca107608 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:03.625662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:03.654843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:03.654843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.654843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.654843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] PATH_INFO: `/v3/projects/d34f1a6845954647b4830afc3193f730` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.655486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34f1a6845954647b4830afc3193f730)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:03.665245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:03.666059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2498b-2661-47f4-90fc-417b1274b6a0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:03.692863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d44ab-7d7c-4c5b-93c5-35220be1925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:03.693566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d44ab-7d7c-4c5b-93c5-35220be1925d admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.693852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d44ab-7d7c-4c5b-93c5-35220be1925d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.694130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d44ab-7d7c-4c5b-93c5-35220be1925d admin admin] PATH_INFO: `/v3/projects/012b24c2fd114e8aa9f8a8fa981c1543/users/7faa29a099c04e2b83e5e41003f6f9a6/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.696197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d44ab-7d7c-4c5b-93c5-35220be1925d admin admin] RBAC: Authorizing `identity:create_grant(project_id=012b24c2fd114e8aa9f8a8fa981c1543, user_id=7faa29a099c04e2b83e5e41003f6f9a6, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:03.698924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d44ab-7d7c-4c5b-93c5-35220be1925d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:03.741678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 891/1774] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:03 2022] PUT /identity/v3/projects/012b24c2fd114e8aa9f8a8fa981c1543/users/7faa29a099c04e2b83e5e41003f6f9a6/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:03.757051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:03.760825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:03.761277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:03.869711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] MFA Rules not processed for user `7faa29a099c04e2b83e5e41003f6f9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:03.875318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:03.968860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34f1a6845954647b4830afc3193f730', 'request_id': 'req-203fdfcb-7667-4dd2-8025-7b4edfe6246d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:03.989225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34f1a6845954647b4830afc3193f730', 'request_id': 'req-203fdfcb-7667-4dd2-8025-7b4edfe6246d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:04.125288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 892/1775] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:04.126600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:04.128900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:04.256917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58229-743a-4ab1-9969-81c45e948c8b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:04.327833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:04.364439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34f1a6845954647b4830afc3193f730', 'request_id': 'req-203fdfcb-7667-4dd2-8025-7b4edfe6246d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:04.393956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] The token cache is being invalidate because project d34f1a6845954647b4830afc3193f730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:04.396610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:04.397514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:04.408935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 884/1776] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:03 2022] DELETE /identity/v3/projects/d34f1a6845954647b4830afc3193f730 => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:04.408935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:04.408935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:04.411597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:04.414044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:04.415509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:04.416129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:04.418297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:04.462836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:04.465319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3568-d94b-4e31-b188-b518d015c257 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:04.540861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 893/1777] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:04.542441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89cb67-77c0-4f42-b20e-9114c0bc3be4 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:04.563333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89cb67-77c0-4f42-b20e-9114c0bc3be4 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:04.564876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89cb67-77c0-4f42-b20e-9114c0bc3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:04.651774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ca89cb67-77c0-4f42-b20e-9114c0bc3be4 None None] MFA Rules not processed for user `4067d48268f84854af4d672ab3f9dc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:04.653039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ca89cb67-77c0-4f42-b20e-9114c0bc3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:04.720504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdfcb-7667-4dd2-8025-7b4edfe6246d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:04.730395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:04.731314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:04.731763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:04.732179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:04.736641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:04.850704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 894/1778] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:04.853697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fb019-a60e-4382-9553-07b3be75b7e6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:04.853697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fb019-a60e-4382-9553-07b3be75b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:04.853697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fb019-a60e-4382-9553-07b3be75b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:04.919094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:04.924037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:04.924227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bb9fb019-a60e-4382-9553-07b3be75b7e6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:04.928187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bb9fb019-a60e-4382-9553-07b3be75b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:04.988308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 885/1779] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:05.117523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:05.124979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:05.179397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 895/1780] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:05.187669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:05.187980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:05.188237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:05.244480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:05.245742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:05.294262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 896/1781] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:05.305458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:05.318971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9573c7-8a13-49ac-a770-53ac301adc7d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:05.324837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:05.325617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:05.326007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:05.326585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:05.328542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:05.329146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:05.331702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:05.366482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 897/1782] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:05.368364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:05.396463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537867d-e981-46da-ae6d-aadff502ace0 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:05.397431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0add65-314c-4e08-88ac-db9f3b62e772 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:05.398232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:05.400215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:05.401118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:05.402807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] PATH_INFO: `/v3/users/1a75206ab6a6432f9687a2deedc38d14` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:05.408986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:05.409831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:05.410247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:05.410620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a75206ab6a6432f9687a2deedc38d14)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:05.410739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:05.411583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:05.417373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:05.432370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:05.500312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:05.648106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 898/1783] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:05 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:05.652692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:05.789887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb867cc-fe48-46eb-a822-a53eaed2ded4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:05.835684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:05.838436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:05.838974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:05.840968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:05.841846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:05.846526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:05.920865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a75206ab6a6432f9687a2deedc38d14'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:06.042023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 899/1784] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:05 2022] POST /identity/v3/users => generated 291 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:06.043388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:06.064785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a75206ab6a6432f9687a2deedc38d14'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:06.068793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a75206ab6a6432f9687a2deedc38d14'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:06.068793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a75206ab6a6432f9687a2deedc38d14'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:06.069112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfb8c4-c810-4147-9ba3-be992aaddd7c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:06.081218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:06.081481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b492147f-7d33-48df-8c03-6b555cf0de14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a75206ab6a6432f9687a2deedc38d14'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:06.082279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:06.083517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:06.083991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:06.085997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:06.097780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:06.123280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 900/1785] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:06.125026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 886/1786] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:05 2022] DELETE /identity/v3/users/1a75206ab6a6432f9687a2deedc38d14 => generated 0 bytes in 1004 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:06.127133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:06.129763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:06.130284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:06.130642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:06.133888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:06.163814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:06.167033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:06.169177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca4816-584f-4f44-8d51-c1e015359df0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:06.303831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:06.309156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:06.310251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:06.311600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:06.315745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:06.318978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:06.358020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 901/1787] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:06.359098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:06.396566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaf205-1636-4b87-85f6-226679d83b15 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:06.414569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dfcc8-84e7-478c-af09-ac20863ca532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:06.416732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dfcc8-84e7-478c-af09-ac20863ca532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:06.417104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dfcc8-84e7-478c-af09-ac20863ca532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:06.418152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dfcc8-84e7-478c-af09-ac20863ca532 admin admin] PATH_INFO: `/v3/projects/1bed83d6b9e14d6297c4188baa47ccb5/users/b3202db2847b439fbca843ed72b5a881/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:06.452584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 887/1788] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:06.471427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dfcc8-84e7-478c-af09-ac20863ca532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bed83d6b9e14d6297c4188baa47ccb5, user_id=b3202db2847b439fbca843ed72b5a881, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:06.478265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dfcc8-84e7-478c-af09-ac20863ca532 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:06.746097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:06.748051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:06.777284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 902/1789] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:06 2022] PUT /identity/v3/projects/1bed83d6b9e14d6297c4188baa47ccb5/users/b3202db2847b439fbca843ed72b5a881/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:06.781469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:06.781966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:06.782831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:06.873480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] MFA Rules not processed for user `b3202db2847b439fbca843ed72b5a881`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:06.880141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:07.018048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cdb04-27ba-4322-aa21-e9acc03c706a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:07.107327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 903/1790] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:07.109461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:07.234138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:07.254940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c84d87-a4a3-49f8-bc88-26aab212a0ec None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:07.274669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:07.281286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:07.281286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:07.281286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:07.281286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:07.287491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:07.288279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:07.288583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:07.288862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:07.296110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:07.326444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:07.331078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:07.358472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 888/1791] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:07.518239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 904/1792] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:07 2022] POST /identity/v3/projects => generated 381 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:07.729367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:07.736989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a6ead-5eee-4de2-94e8-5ed3449b31f6 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:07.841370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:07.843336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:07.938196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:07.938859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:07.939353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:07.942424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:07.943945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:07.953479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:08.080952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 889/1793] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:07 2022] POST /identity/v3/users => generated 297 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:08.082799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866051c-4062-4a55-89c9-7a355b467746 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:08.089710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:08.096114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:08.097645 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:08.098248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:08.098733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] PATH_INFO: `/v3/users/429336a1ae734517ad86bd576e9e24c1` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:08.101189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=429336a1ae734517ad86bd576e9e24c1)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:08.111550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abf436-0ab4-4895-a889-dd26cbe4f0a7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:08.116803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:08.165048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:08.165717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:08.165997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:08.166258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:08.167026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:08.177580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:08.203421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 890/1794] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:08.208785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:08.256453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113d5cd-1273-455b-a74c-78a879b3af0d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:08.284251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:08.287895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:08.293064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:08.294320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] PATH_INFO: `/v3/projects/5113dfa8fa7d43af97eee32db18ceda3/users/bb66bdcfdcde4ea7ad5e87c4a325b884/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:08.352055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5113dfa8fa7d43af97eee32db18ceda3, user_id=bb66bdcfdcde4ea7ad5e87c4a325b884, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:08.358023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:08.395572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429336a1ae734517ad86bd576e9e24c1'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:08.451196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 891/1795] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:08 2022] PUT /identity/v3/projects/5113dfa8fa7d43af97eee32db18ceda3/users/bb66bdcfdcde4ea7ad5e87c4a325b884/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:08.451196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:08.453953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:08.492926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429336a1ae734517ad86bd576e9e24c1'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:08.493907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429336a1ae734517ad86bd576e9e24c1'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:08.494704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429336a1ae734517ad86bd576e9e24c1'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:08.510496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429336a1ae734517ad86bd576e9e24c1'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:08.642133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 905/1796] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:07 2022] DELETE /identity/v3/users/429336a1ae734517ad86bd576e9e24c1 => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:08.645112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:08.650526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:08.768323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629a10-7271-46b7-9640-f0127ea9ec23 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:08.877039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c650d0-9e43-4ba0-b642-1c6e20fa34b6 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:08.896567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5498ca-c8a0-4062-ad06-bf1d3212528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:08.897466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5498ca-c8a0-4062-ad06-bf1d3212528d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:08.897898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5498ca-c8a0-4062-ad06-bf1d3212528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:08.898329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5498ca-c8a0-4062-ad06-bf1d3212528d admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:08.899431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5498ca-c8a0-4062-ad06-bf1d3212528d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:08.901232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5498ca-c8a0-4062-ad06-bf1d3212528d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:08.938986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 906/1797] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:08.950148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:08.950440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:08.951493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:08.962207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:08.963293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:08.963749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:08.964176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] PATH_INFO: `/v3/users/5bda445222c44eb09b215e27fe40a0f8` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:08.967234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bda445222c44eb09b215e27fe40a0f8)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:08.975672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:09.007834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:09.009392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:09.039926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 907/1798] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:09.041624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:09.063237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a4a36-3a61-4109-a91c-2dedae23cd20 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:09.076102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:09.077585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:09.078072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:09.078537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] PATH_INFO: `/v3/projects/5113dfa8fa7d43af97eee32db18ceda3/users/bb66bdcfdcde4ea7ad5e87c4a325b884/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:09.081020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5113dfa8fa7d43af97eee32db18ceda3, user_id=bb66bdcfdcde4ea7ad5e87c4a325b884, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:09.084649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:09.122797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 908/1799] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:09 2022] PUT /identity/v3/projects/5113dfa8fa7d43af97eee32db18ceda3/users/bb66bdcfdcde4ea7ad5e87c4a325b884/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:09.124201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:09.127154 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:09.273582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68df3a2-90cf-4d4f-bfa2-1169c6e0c247 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:09.292373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:09.296829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:09.296829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:09.296829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] PATH_INFO: `/v3/projects/c89cf0f3b72f47b391dcef94d15fb133` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:09.296829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89cf0f3b72f47b391dcef94d15fb133)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:09.296829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:09.423125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89cf0f3b72f47b391dcef94d15fb133', 'request_id': 'req-6da7fd89-e591-4532-9867-58a30c7e9015'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.433024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89cf0f3b72f47b391dcef94d15fb133', 'request_id': 'req-6da7fd89-e591-4532-9867-58a30c7e9015'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.452975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bda445222c44eb09b215e27fe40a0f8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.465852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:09.532093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89cf0f3b72f47b391dcef94d15fb133', 'request_id': 'req-6da7fd89-e591-4532-9867-58a30c7e9015'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.551337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] The token cache is being invalidate because project c89cf0f3b72f47b391dcef94d15fb133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:09.551650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.552598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.554512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 909/1800] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:09 2022] DELETE /identity/v3/projects/c89cf0f3b72f47b391dcef94d15fb133 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:09.555478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:09.558909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:09.616676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bda445222c44eb09b215e27fe40a0f8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.617966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bda445222c44eb09b215e27fe40a0f8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.619052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bda445222c44eb09b215e27fe40a0f8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.637126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c2b21b9-a794-4193-98ad-82fbc1165714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bda445222c44eb09b215e27fe40a0f8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:09.684021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7fd89-e591-4532-9867-58a30c7e9015 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:09.696319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 892/1801] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:08 2022] DELETE /identity/v3/users/5bda445222c44eb09b215e27fe40a0f8 => generated 0 bytes in 1247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:09.698131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e71303-40df-421f-a01a-dd802402a96b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:09.698429 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e71303-40df-421f-a01a-dd802402a96b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:09.698710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e71303-40df-421f-a01a-dd802402a96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:09.790221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a7e71303-40df-421f-a01a-dd802402a96b None None] MFA Rules not processed for user `bb66bdcfdcde4ea7ad5e87c4a325b884`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:09.800087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a7e71303-40df-421f-a01a-dd802402a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:09.804129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:09.804393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:09.804523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:09.804808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:09.807003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:09.936325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:09.939204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:09.958882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 910/1802] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:09.965956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:09.965956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:09.970390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 893/1803] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:09.976711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:09.976711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:09.977083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:10.003683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:10.005714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:10.017399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa06ba-b321-4655-966e-2ea610f3ab73 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:10.017834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 894/1804] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:10.020810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:10.023109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:10.030303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:10.031001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:10.031339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:10.031613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] PATH_INFO: `/v3/users/7faa29a099c04e2b83e5e41003f6f9a6` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:10.033230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7faa29a099c04e2b83e5e41003f6f9a6)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:10.037222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:10.167888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730cc24-219e-4bd8-a4d6-f0ddb8a89d6f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:10.172624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:10.173652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:10.173652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:10.173853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:10.237823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:10.243822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:10.275498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 895/1805] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:10.280736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:10.284747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:10.308728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bdf92-a777-4668-8b59-df63638e14bb service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:10.326337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1fabd-1431-465a-aca8-e03e6f7a48e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:10.326337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1fabd-1431-465a-aca8-e03e6f7a48e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:10.326337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1fabd-1431-465a-aca8-e03e6f7a48e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:10.326337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1fabd-1431-465a-aca8-e03e6f7a48e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:10.336473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1fabd-1431-465a-aca8-e03e6f7a48e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:10.338266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1fabd-1431-465a-aca8-e03e6f7a48e3 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:10.353471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 896/1806] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:10.355240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e9250-f422-41de-9125-87df04c20037 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:10.355620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e9250-f422-41de-9125-87df04c20037 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:10.356032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e9250-f422-41de-9125-87df04c20037 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:10.357899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 897/1807] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:16:10.387100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:10.389239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:10.391112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:10.402365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7faa29a099c04e2b83e5e41003f6f9a6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:10.427558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:10.428951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:10.542782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7faa29a099c04e2b83e5e41003f6f9a6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:10.543583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7faa29a099c04e2b83e5e41003f6f9a6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:10.544296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7faa29a099c04e2b83e5e41003f6f9a6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:10.558496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7faa29a099c04e2b83e5e41003f6f9a6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:10.605779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 911/1808] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:09 2022] DELETE /identity/v3/users/7faa29a099c04e2b83e5e41003f6f9a6 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:10.606127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 898/1809] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:10.824697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:10.826778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:11.016237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4376b13-fda5-4df5-95cc-d6a459adde8b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:11.026065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:11.027013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:11.027542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:11.031770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:11.048552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:11.056267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:11.060552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:11.085567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 912/1810] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:11.317707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:11.320228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:11.350696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e889b3a-2609-48e9-9840-90436aef8f73 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:11.476206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:11.494771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:11.494771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:11.494771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] PATH_INFO: `/v3/users/b3202db2847b439fbca843ed72b5a881` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:11.494771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3202db2847b439fbca843ed72b5a881)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:11.494771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:11.724775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:11.737710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:11.794839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28a6f-0660-494e-8c69-aa6efa98e4f3 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:11.821489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:11.822221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:11.822540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:11.822855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] PATH_INFO: `/v3/projects/fd5b0cd2ca6247ee8d45c28f1c01e0d5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:11.841217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5b0cd2ca6247ee8d45c28f1c01e0d5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:11.843391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:12.018747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3202db2847b439fbca843ed72b5a881'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.101542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5b0cd2ca6247ee8d45c28f1c01e0d5', 'request_id': 'req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.122508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5b0cd2ca6247ee8d45c28f1c01e0d5', 'request_id': 'req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.127280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3202db2847b439fbca843ed72b5a881'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.128862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3202db2847b439fbca843ed72b5a881'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.129831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3202db2847b439fbca843ed72b5a881'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.144106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3202db2847b439fbca843ed72b5a881'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.190666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 899/1811] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:11 2022] DELETE /identity/v3/users/b3202db2847b439fbca843ed72b5a881 => generated 0 bytes in 871 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:12.192103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:12.202791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:12.241001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5b0cd2ca6247ee8d45c28f1c01e0d5', 'request_id': 'req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.255899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] The token cache is being invalidate because project fd5b0cd2ca6247ee8d45c28f1c01e0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:12.256212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.258538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7e5833c9-6c5c-4dba-93bb-2b5a6d0d2a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.261900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 913/1812] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:11 2022] DELETE /identity/v3/projects/fd5b0cd2ca6247ee8d45c28f1c01e0d5 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:12.268041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:12.275674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:12.275674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:12.297276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:12.299575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:12.420112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061acc2-87af-4766-b8d7-1f3176e21516 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:12.427696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:12.428698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:12.429324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:12.429889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] PATH_INFO: `/v3/projects/c55232d1d89247378fac6a2d591c2930` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:12.431372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55232d1d89247378fac6a2d591c2930)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:12.434340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:12.465262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 914/1813] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:12.465262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:12.472606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:12.509542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8e858-c1fb-440c-b7d2-1dfc03a2d758 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:12.547248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:12.547784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:12.547973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:12.548259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] PATH_INFO: `/v3/users/a9f6ec3fe6064b7a8d0bc10b37fbd1d0` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:12.558003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f6ec3fe6064b7a8d0bc10b37fbd1d0)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:12.560031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:12.808261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55232d1d89247378fac6a2d591c2930', 'request_id': 'req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.819777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55232d1d89247378fac6a2d591c2930', 'request_id': 'req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.927295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55232d1d89247378fac6a2d591c2930', 'request_id': 'req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.944717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] The token cache is being invalidate because project c55232d1d89247378fac6a2d591c2930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:12.945022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.950409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:12.967000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.970246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 900/1814] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:12 2022] DELETE /identity/v3/projects/c55232d1d89247378fac6a2d591c2930 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:12.978136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:12.978248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f6ec3fe6064b7a8d0bc10b37fbd1d0'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:12.981880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.051409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fcaca-02e2-4bc6-803c-5f7b6e4d6122 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.056196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:13.056936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.057166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.057440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] PATH_INFO: `/v3/projects/012b24c2fd114e8aa9f8a8fa981c1543` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.058635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012b24c2fd114e8aa9f8a8fa981c1543)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:13.060718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:13.063593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f6ec3fe6064b7a8d0bc10b37fbd1d0'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.064228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f6ec3fe6064b7a8d0bc10b37fbd1d0'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.064843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f6ec3fe6064b7a8d0bc10b37fbd1d0'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.074234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f6ec3fe6064b7a8d0bc10b37fbd1d0'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.102555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 915/1815] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:12 2022] DELETE /identity/v3/users/a9f6ec3fe6064b7a8d0bc10b37fbd1d0 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:13.104466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.106465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.206246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1247dc-7560-4467-b242-d1db6a3d4a0f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.225621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012b24c2fd114e8aa9f8a8fa981c1543', 'request_id': 'req-1dda2d68-ace7-429a-86a0-4c0da23c9d58'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.238648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012b24c2fd114e8aa9f8a8fa981c1543', 'request_id': 'req-1dda2d68-ace7-429a-86a0-4c0da23c9d58'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.284417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:13.287579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.288162 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.289019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.292536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.315543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012b24c2fd114e8aa9f8a8fa981c1543', 'request_id': 'req-1dda2d68-ace7-429a-86a0-4c0da23c9d58'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.341431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] The token cache is being invalidate because project 012b24c2fd114e8aa9f8a8fa981c1543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:13.341960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.342814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.344839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 901/1816] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:12 2022] DELETE /identity/v3/projects/012b24c2fd114e8aa9f8a8fa981c1543 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:13.345890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.349197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.380533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:13.386962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc0e4b-ec56-4835-91d2-accd465c8900 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:13.395591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 916/1817] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:13.400357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81aec-1080-4cb4-bca6-ae5a7a227985 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.400663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81aec-1080-4cb4-bca6-ae5a7a227985 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.400962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81aec-1080-4cb4-bca6-ae5a7a227985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.427453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ade81aec-1080-4cb4-bca6-ae5a7a227985 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:13.428921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ade81aec-1080-4cb4-bca6-ae5a7a227985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.452609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 917/1818] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:13.452609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154217a-a855-4a95-b037-947665cd37bc None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.452609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154217a-a855-4a95-b037-947665cd37bc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.452609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154217a-a855-4a95-b037-947665cd37bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.469880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0154217a-a855-4a95-b037-947665cd37bc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:13.471229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0154217a-a855-4a95-b037-947665cd37bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.492579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 918/1819] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:13.497313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154217a-a855-4a95-b037-947665cd37bc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.508003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154217a-a855-4a95-b037-947665cd37bc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.509861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda2d68-ace7-429a-86a0-4c0da23c9d58 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.513372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:13.513986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.514268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.514543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.515229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:13.515343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:13.515540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:13.515924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.516214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.516483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] PATH_INFO: `/v3/users/f6edda3e45a54a7f9ec9bce0d311636a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.517319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:13.517964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6edda3e45a54a7f9ec9bce0d311636a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:13.521116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:13.536786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 919/1820] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:13.537915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.548656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3fd8c-f775-47d3-971c-7fca6543b42f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.554770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:13.555448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.555722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.555995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.556631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:13.558570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:13.685475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 920/1821] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:13 2022] POST /identity/v3/projects => generated 401 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:13.686803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.749536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6edda3e45a54a7f9ec9bce0d311636a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.780594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6447e09-fee1-49e1-b2a6-27d9d7aab5cb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.785954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:13.786684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:13.787023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:13.787395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:13.788093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:13.790232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:13.835689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6edda3e45a54a7f9ec9bce0d311636a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.836318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6edda3e45a54a7f9ec9bce0d311636a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.836873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:13.837021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6edda3e45a54a7f9ec9bce0d311636a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.845941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6edda3e45a54a7f9ec9bce0d311636a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:13.874749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 921/1822] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:13 2022] POST /identity/v3/users => generated 301 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:13.875983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.887452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:13.905540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 902/1823] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:13 2022] DELETE /identity/v3/users/f6edda3e45a54a7f9ec9bce0d311636a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:13.917264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4e84a-7592-4386-b182-3fa713d6f8aa admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:13.996506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:13.998377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.012407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.013189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.013566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.014090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.014841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.021433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.045399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 922/1824] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:14.055265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.069981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84361a2a-63c4-4f3d-8777-b3156108838e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.075066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.075708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.075992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.076266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.076934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.078537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.092302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 923/1825] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:14.097025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.103899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ead5ab-8b61-4390-8924-a2366d630484 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.106902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179e25-3060-4e6a-a9e1-c2ec41a7e2b8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.110427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.111155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.111537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.112434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] PATH_INFO: `/v3/projects/36228d16965441a6a77f35d1a70cbd26` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.113730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36228d16965441a6a77f35d1a70cbd26)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.114947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.115802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.118415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.118524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.118524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] PATH_INFO: `/v3/projects/f120db82790e4eda957d951a7d269269/users/46b4e6260d064b2abfb471e06cd4686a/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.134522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f120db82790e4eda957d951a7d269269, user_id=46b4e6260d064b2abfb471e06cd4686a, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.137472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.238198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 924/1826] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:14 2022] PUT /identity/v3/projects/f120db82790e4eda957d951a7d269269/users/46b4e6260d064b2abfb471e06cd4686a/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:14.243321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.253236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.276981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36228d16965441a6a77f35d1a70cbd26', 'request_id': 'req-b876cd23-4e97-4ced-9bea-46dbde12314a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.284899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36228d16965441a6a77f35d1a70cbd26', 'request_id': 'req-b876cd23-4e97-4ced-9bea-46dbde12314a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.322697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f154a1-ec78-4b59-89e2-e031631d326a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.333189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:14.364575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36228d16965441a6a77f35d1a70cbd26', 'request_id': 'req-b876cd23-4e97-4ced-9bea-46dbde12314a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.375395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] The token cache is being invalidate because project 36228d16965441a6a77f35d1a70cbd26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:14.375766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.376479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b876cd23-4e97-4ced-9bea-46dbde12314a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.378089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 903/1827] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:13 2022] DELETE /identity/v3/projects/36228d16965441a6a77f35d1a70cbd26 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:14.378738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.379394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ee010-7193-4a57-8632-05ccca98e1c4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.379499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.379676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ee010-7193-4a57-8632-05ccca98e1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.379772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.379947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ee010-7193-4a57-8632-05ccca98e1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.380027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] PATH_INFO: `/v3/projects/1bed83d6b9e14d6297c4188baa47ccb5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.381928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bed83d6b9e14d6297c4188baa47ccb5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.383679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.411153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d72ee010-7193-4a57-8632-05ccca98e1c4 None None] MFA Rules not processed for user `46b4e6260d064b2abfb471e06cd4686a`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:14.412588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d72ee010-7193-4a57-8632-05ccca98e1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.486323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 904/1828] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:14.487496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.487948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.488231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.498165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bed83d6b9e14d6297c4188baa47ccb5', 'request_id': 'req-fd8745d7-15be-40d3-babe-79803091c96d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.504900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bed83d6b9e14d6297c4188baa47ccb5', 'request_id': 'req-fd8745d7-15be-40d3-babe-79803091c96d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.508222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:14.509462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.568934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 905/1829] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:14.569786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.571082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.582331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807702d7-0a20-4bfb-9383-a939b6a181ef None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.587052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.587697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.587988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.588264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] PATH_INFO: `/v3/projects/9bb601e9f14e440dae0b8cd2df8ba22c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.589402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb601e9f14e440dae0b8cd2df8ba22c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.589517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bed83d6b9e14d6297c4188baa47ccb5', 'request_id': 'req-fd8745d7-15be-40d3-babe-79803091c96d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.591334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.602331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] The token cache is being invalidate because project 1bed83d6b9e14d6297c4188baa47ccb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:14.602702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.603461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.605140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 925/1830] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:14 2022] DELETE /identity/v3/projects/1bed83d6b9e14d6297c4188baa47ccb5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:14.607213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.607368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.620635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8745d7-15be-40d3-babe-79803091c96d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.625296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698d14-66b8-468f-b142-85c1680d7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.625894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698d14-66b8-468f-b142-85c1680d7ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.626176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698d14-66b8-468f-b142-85c1680d7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.626452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698d14-66b8-468f-b142-85c1680d7ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.627114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698d14-66b8-468f-b142-85c1680d7ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.629084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698d14-66b8-468f-b142-85c1680d7ee9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.685416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb601e9f14e440dae0b8cd2df8ba22c', 'request_id': 'req-3265b32f-6987-482b-b851-1b52d3b7fc48'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.691575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb601e9f14e440dae0b8cd2df8ba22c', 'request_id': 'req-3265b32f-6987-482b-b851-1b52d3b7fc48'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.699651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 926/1831] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:14 2022] POST /identity/v3/projects => generated 399 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:14.701099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.701391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.701670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.720448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:14.722144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.772361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb601e9f14e440dae0b8cd2df8ba22c', 'request_id': 'req-3265b32f-6987-482b-b851-1b52d3b7fc48'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.784423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 927/1832] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:14.784423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.784423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.785564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] The token cache is being invalidate because project 9bb601e9f14e440dae0b8cd2df8ba22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:14.785897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.786680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:14.788237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 906/1833] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:14 2022] DELETE /identity/v3/projects/9bb601e9f14e440dae0b8cd2df8ba22c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:14.789215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.790973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.795479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e41bb3-88c7-4780-9cae-8308ccaabc79 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.800957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.801628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.801967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.802297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] PATH_INFO: `/v3/users/bb66bdcfdcde4ea7ad5e87c4a325b884` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.803608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb66bdcfdcde4ea7ad5e87c4a325b884)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.805611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.856098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b32f-6987-482b-b851-1b52d3b7fc48 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:14.861171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:14.862428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:14.862712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:14.862983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:14.863655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:14.865685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:14.977352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 907/1834] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:14 2022] POST /identity/v3/users => generated 306 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:14.977352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:14.979078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:14.992051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310e01b-50dc-4d6e-8c9b-927fe4fcca77 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.000804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.001426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.003271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.003271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.003271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:15.003271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.004763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4847974-6d2d-47bb-b004-bdc96eb713a4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.015659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb66bdcfdcde4ea7ad5e87c4a325b884'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:15.023546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 908/1835] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:15.023546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.023546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.023546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.023969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:15.044393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:15.045821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:15.054224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 909/1836] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:15.055148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.065391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aba8b8-b351-4988-8e56-07fa6490ae91 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.070101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.070712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.071189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.071470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.072120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.073721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.085070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 910/1837] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:15.086136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.091861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb66bdcfdcde4ea7ad5e87c4a325b884'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:15.093195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb66bdcfdcde4ea7ad5e87c4a325b884'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:15.093928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb66bdcfdcde4ea7ad5e87c4a325b884'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:15.099474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3917-363f-4c20-a714-054ac29546a6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.104385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.105687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bdb61a74-acf2-46d8-8cee-b29682b5e1ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb66bdcfdcde4ea7ad5e87c4a325b884'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:15.106077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.106505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.107068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.107746 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.109655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.140796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 928/1838] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:14 2022] DELETE /identity/v3/users/bb66bdcfdcde4ea7ad5e87c4a325b884 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:15.142096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.142599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.142891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.165229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:15.166900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:15.211394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 911/1839] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:15.213153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.217247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:15.243464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 929/1840] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:15.244435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.246377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:15.258746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44093c24-f858-4e78-b614-56bf8af6e205 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.305059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4249bc9-ca4d-4c2f-969b-3194204d81b3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.311418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.312088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.312366 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.312635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] PATH_INFO: `/v3/projects/1767da9d5c944adc897c682a88bd0f30/users/a0ba67173af4465e8090e129323ff795/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.328434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1767da9d5c944adc897c682a88bd0f30, user_id=a0ba67173af4465e8090e129323ff795, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.331007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.331007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.331007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.331007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.331522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.332141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.334472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.350595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:15.414251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 912/1841] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:15 2022] PUT /identity/v3/projects/1767da9d5c944adc897c682a88bd0f30/users/a0ba67173af4465e8090e129323ff795/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:15.417441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.484630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 930/1842] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:15.486110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.498775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5275a-4e89-4058-9dcb-883e3401cf81 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.546737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ca559-bdb1-4303-8d2d-483fa2a9479f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.553901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.554359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.554649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.554928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.555621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:15.555932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.557729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.576165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 913/1843] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:15.577045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.590111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120e358-aea2-426c-9d27-4a3cd7b608f6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.593773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.594609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.594896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.595188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.595670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.595850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.597155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.597520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.597904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.598499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.598690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.600662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.616417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 931/1844] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:15.616782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 914/1845] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:15.617332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.617937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.628149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e6a53-dd37-49a9-9a73-594f695a6aaf admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.632979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.633593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.634019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.634137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] PATH_INFO: `/v3/projects/1767da9d5c944adc897c682a88bd0f30/users/a0ba67173af4465e8090e129323ff795/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.636209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1767da9d5c944adc897c682a88bd0f30, user_id=a0ba67173af4465e8090e129323ff795, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.639031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.643851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238c854-680d-43d1-b95f-3e83114d9bc9 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.649828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8988a-b7c9-4d88-b50b-89b1fc3ac689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.650313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8988a-b7c9-4d88-b50b-89b1fc3ac689 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.651029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8988a-b7c9-4d88-b50b-89b1fc3ac689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.651029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8988a-b7c9-4d88-b50b-89b1fc3ac689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.651581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8988a-b7c9-4d88-b50b-89b1fc3ac689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.653625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8988a-b7c9-4d88-b50b-89b1fc3ac689 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.661032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 915/1846] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:15 2022] PUT /identity/v3/projects/1767da9d5c944adc897c682a88bd0f30/users/a0ba67173af4465e8090e129323ff795/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:15.662037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.758165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 932/1847] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:15.759696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.759979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.760253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.771962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08381d3-d07a-4ee0-afa9-080a15b3f0a5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.802149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] MFA Rules not processed for user `a0ba67173af4465e8090e129323ff795`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:15.804173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:15.881380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.882020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.882304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.882579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.887353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.890444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.916364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 933/1848] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:15.916364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.919752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 916/1849] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:15.919752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:15.923380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:15.941804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774d1b-8bf3-4186-b851-b3c65351f70f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.950712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.954957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.954957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.954957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.954957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.955305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b39d8-4941-4b00-ac9e-fa9baf76da52 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:15.956687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:15.966995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:15.967693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:15.968035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:15.968317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] PATH_INFO: `/v3/projects/5113dfa8fa7d43af97eee32db18ceda3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:15.970259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5113dfa8fa7d43af97eee32db18ceda3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:15.978097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.224603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 934/1850] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:15 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:16.225661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.238649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3975-37d5-4f29-8d43-3af518195a72 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:16.253024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:16.253721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.254001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.254317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] PATH_INFO: `/v3/projects/dbe7e9190ae24a5695c4c108b88d394d/users/3e909b879d674ecf9297b968c9a4d988/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.289129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5113dfa8fa7d43af97eee32db18ceda3', 'request_id': 'req-599c6a74-c398-493c-a778-aba06bf06092'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:16.298575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe7e9190ae24a5695c4c108b88d394d, user_id=3e909b879d674ecf9297b968c9a4d988, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:16.298896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5113dfa8fa7d43af97eee32db18ceda3', 'request_id': 'req-599c6a74-c398-493c-a778-aba06bf06092'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:16.305016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.399344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:16.402294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5113dfa8fa7d43af97eee32db18ceda3', 'request_id': 'req-599c6a74-c398-493c-a778-aba06bf06092'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:16.406067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 935/1851] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:16 2022] PUT /identity/v3/projects/dbe7e9190ae24a5695c4c108b88d394d/users/3e909b879d674ecf9297b968c9a4d988/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:16.407639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.410512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.418881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] The token cache is being invalidate because project 5113dfa8fa7d43af97eee32db18ceda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:16.419773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:16.420602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:16.422629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 917/1852] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:15 2022] DELETE /identity/v3/projects/5113dfa8fa7d43af97eee32db18ceda3 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:16.424060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.426055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.487749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c954b-f149-4416-87ca-7ee7e4e01e2e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:16.492079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:16.492722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.493069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.493408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.495269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.514641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6a74-c398-493c-a778-aba06bf06092 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:16.603452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:16.605196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065c032-babd-4992-90a6-68bb48ffb2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:16.605337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d0446-61d2-4262-a6c5-73165cc4e2a0 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.605820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065c032-babd-4992-90a6-68bb48ffb2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.606279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065c032-babd-4992-90a6-68bb48ffb2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.606696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065c032-babd-4992-90a6-68bb48ffb2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.607465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065c032-babd-4992-90a6-68bb48ffb2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:16.610862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065c032-babd-4992-90a6-68bb48ffb2d2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.619006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 936/1853] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:16 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:16.621902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.622197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.622486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.638309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 918/1854] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:16.639850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.643920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.643920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.656370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] MFA Rules not processed for user `3e909b879d674ecf9297b968c9a4d988`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:16.657920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.664573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:16.665982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.675601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 919/1855] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:16.682794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.693795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:16.694036 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cde144-2f68-490d-86f9-7f239c20f9af None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:16.699901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fe9d5-0059-4bde-b66e-cb3819196ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:16.702168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe9d5-0059-4bde-b66e-cb3819196ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.702168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe9d5-0059-4bde-b66e-cb3819196ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.702168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe9d5-0059-4bde-b66e-cb3819196ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.702405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fe9d5-0059-4bde-b66e-cb3819196ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:16.704824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fe9d5-0059-4bde-b66e-cb3819196ef8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.724064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 920/1856] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:16.726856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.727345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.730146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.744598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 937/1857] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:16.744598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.754810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:16.759283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.761482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a338323-ef4b-46b9-a2ef-0b185ab2c7bd None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:16.767051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 921/1858] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:16.767769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:16.768601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.768888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.769168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] PATH_INFO: `/v3/projects/b598156982eb437b9abbe7c3516dec91/users/daf1691634ae4be6ae5c6e2e127c3154/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.771322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.772719 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:16.786160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a06179-8d1c-48e4-851f-c1f262d21932 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:16.791487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b598156982eb437b9abbe7c3516dec91, user_id=daf1691634ae4be6ae5c6e2e127c3154, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:16.791675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:16.792313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:16.792664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:16.793007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:16.793732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:16.795435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.795820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:16.947232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 938/1859] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:16 2022] PUT /identity/v3/projects/b598156982eb437b9abbe7c3516dec91/users/daf1691634ae4be6ae5c6e2e127c3154/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:16.948516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.963583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 922/1860] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:16 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:16.964737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:16.967967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:17.030246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ed55-7e0f-4429-8f02-ffe42bc719fa admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.042135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9b15-55c7-465e-884a-630b50fa0ab1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.045945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.046566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.055926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.055926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.055926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:17.100277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.101002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.101379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.101720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.102560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:17.102944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.104597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc114d6-a60d-4444-a6f7-3a675e6e4be2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.114028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.115517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.121010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 939/1861] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:17.121010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.121010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.121010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.126409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 923/1862] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:16 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:17.127571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.141463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf68d1c-1fb8-4f24-a73c-16070c8d2507 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.148647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.148647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.148647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.148647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.149335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.151314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.165468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] MFA Rules not processed for user `daf1691634ae4be6ae5c6e2e127c3154`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:17.166805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:17.242144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 940/1863] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:17.243011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.254885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9439ef2-4e75-41ab-b6a3-7425b798d284 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.260296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.261041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.261383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.262186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.262874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.264583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.271506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 924/1864] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:17 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:17.274624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.288239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c950a67-9983-4711-a1d8-b04125f4101a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.294672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.295439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.296178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.296178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.297101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.299345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.424179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 925/1865] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:17 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:17.424179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.447847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 941/1866] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:17 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:17.448939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.541400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c7957-8655-44d6-a9b0-323ef916c7be admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.546607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.548322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.548322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.548322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.548587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.550165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.561891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c6a25-a094-4b80-b003-ecd16bf77e53 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.562965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 926/1867] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:17.564171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.576774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.578430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.578688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.578968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc09254-0baa-463b-8bbe-fcd442ec4b4c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.580067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.584159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.585801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.587113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.587506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.587818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.588708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.589611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.591818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.601452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:17.690669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 927/1868] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:17 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:17.691793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.718375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdb2d6-43e1-48ab-8790-ab878df48de1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.723445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.724115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.724401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.724669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.725323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.727059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.739527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 928/1869] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:17.740526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.745451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 942/1870] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:17 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:17.747344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.754780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cd5e7-e640-4b53-99cf-7bfe464908da admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.762930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2443a3-d614-4d63-b0de-2ac429ea2519 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.771525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.773402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.773823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.774234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.774458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.774743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.775593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.775721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] PATH_INFO: `/v3/projects/bfd6ef0a30e94211a74cf478f7675592/users/801173c9a43f40a8821d954946f90760/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.776251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.778328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.795543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 929/1871] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:17.795543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.804000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd6ef0a30e94211a74cf478f7675592, user_id=801173c9a43f40a8821d954946f90760, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.804732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9c56f-ffc6-4ff1-a60b-17de9e7a14ab admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.807946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.809704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.810318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.810918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.811493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.812208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.813887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.827685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 930/1872] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:17.828628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.844232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8507d38-8f42-472b-a6ab-9b577c134fc1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.853632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.853632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.853632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.853632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.854337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.857836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.879723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 931/1873] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:17.882306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.896712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38889c0-d0c9-4e1c-98ce-3455bff8a71f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:17.903882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:17.904475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:17.905276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:17.905388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:17.905955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:17.907851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:17.923724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 943/1874] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:17 2022] PUT /identity/v3/projects/bfd6ef0a30e94211a74cf478f7675592/users/801173c9a43f40a8821d954946f90760/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:17.923724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.926130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 932/1875] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:17.927078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:17.953467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c450b0-3c1b-4727-bdfa-bce09d094b64 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.023184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af697e51-f2b5-4060-ba59-b35f837b66b3 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.043262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44555f3-56c3-477e-bb84-349320ab293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.043900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44555f3-56c3-477e-bb84-349320ab293e admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.045929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44555f3-56c3-477e-bb84-349320ab293e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.046394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44555f3-56c3-477e-bb84-349320ab293e admin admin] PATH_INFO: `/v3/projects/bd469d812fad4dd18a0c73980f756e8d/users/cb64ac86a6bc4699ba9e65934efa056f/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.093331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44555f3-56c3-477e-bb84-349320ab293e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd469d812fad4dd18a0c73980f756e8d, user_id=cb64ac86a6bc4699ba9e65934efa056f, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.103770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44555f3-56c3-477e-bb84-349320ab293e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.113136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7221-bdbc-469d-8d50-c50f647a460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.113814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7221-bdbc-469d-8d50-c50f647a460a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.114092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7221-bdbc-469d-8d50-c50f647a460a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.114364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7221-bdbc-469d-8d50-c50f647a460a admin admin] PATH_INFO: `/v3/projects/25a77bb4e1f541ca9e74fceaa6403fba/users/4524e23f01cf465d9996a0d359958331/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.136874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e7221-bdbc-469d-8d50-c50f647a460a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a77bb4e1f541ca9e74fceaa6403fba, user_id=4524e23f01cf465d9996a0d359958331, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.140033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e7221-bdbc-469d-8d50-c50f647a460a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.186183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 944/1876] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:17 2022] PUT /identity/v3/projects/bd469d812fad4dd18a0c73980f756e8d/users/cb64ac86a6bc4699ba9e65934efa056f/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:18.187392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85528a29-258e-410c-8af4-72092e57d85c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.187681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85528a29-258e-410c-8af4-72092e57d85c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.187956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85528a29-258e-410c-8af4-72092e57d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.219956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-85528a29-258e-410c-8af4-72092e57d85c None None] MFA Rules not processed for user `801173c9a43f40a8821d954946f90760`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:18.221812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-85528a29-258e-410c-8af4-72092e57d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:18.257156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 933/1877] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:17 2022] PUT /identity/v3/projects/25a77bb4e1f541ca9e74fceaa6403fba/users/4524e23f01cf465d9996a0d359958331/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:18.258695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531eb15-1962-4523-bc6d-06b4c20a65d6 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.258984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531eb15-1962-4523-bc6d-06b4c20a65d6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.259555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531eb15-1962-4523-bc6d-06b4c20a65d6 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.261187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 934/1878] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:16:18.262725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c810fea-7a38-4508-9fd4-5f114f35d444 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.263006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c810fea-7a38-4508-9fd4-5f114f35d444 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.263431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c810fea-7a38-4508-9fd4-5f114f35d444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.290063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6c810fea-7a38-4508-9fd4-5f114f35d444 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:18.292465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6c810fea-7a38-4508-9fd4-5f114f35d444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:18.384782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 945/1879] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:18.403338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 935/1880] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:18.403338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.403338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.404469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.404555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.404631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.404631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.422458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:18.426279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:18.443407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] MFA Rules not processed for user `4524e23f01cf465d9996a0d359958331`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:18.445231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:18.525015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 936/1881] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:18.526361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.539776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3b6a3-0d38-4752-ad47-45ee8d5697a5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.546057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.546702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.548362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.548666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.549308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 946/1882] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:18.553152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.553152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.554910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.556915 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:18.637820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9a882-83c6-4040-9c44-c14a28a97621 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.641881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.643663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.643954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.644234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.659205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 937/1883] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:18.661470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.662681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.663281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:18.664802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.677866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:18.681149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 947/1884] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:18.681994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.697100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b93e42-3932-4def-a1e1-dba9e6c6e824 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.765256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.765864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.766151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.766482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.767129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.773951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.774091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed26422-4b35-44dc-be36-441621e35a01 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.778398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.779019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.779555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.780305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.789770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.791401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.802807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 938/1885] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:18.803641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.814031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37017-a827-4356-8785-2f45e1f5f965 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.820410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.820981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.821284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.821539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.822180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.824192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.862398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 948/1886] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:18.866723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.927052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 939/1887] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:18.929589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.936613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:18.951754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bed7-904c-4329-ac26-1e6f6c28d010 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.951938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e398-3ba7-45fc-ad93-860fbe750282 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:18.958025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.958420 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:18.958752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.959051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.959360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.960025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.960148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:18.960635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:18.961045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:18.961833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.961977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:18.965554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:18.981062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 940/1888] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:18.982097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:18.998193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9558ec-e9ec-4fd9-9736-52a1f78ff3fe admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.008874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.010281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.010722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.011868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] PATH_INFO: `/v3/projects/1c4c2ad68a784b19a339e87595635ae1/users/805dfdb1a5d742069c79d48c7d8dab8f/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.040413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4c2ad68a784b19a339e87595635ae1, user_id=805dfdb1a5d742069c79d48c7d8dab8f, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.043233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.094815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 949/1889] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:18 2022] POST /identity/v3/users => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:19.098844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.136399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 941/1890] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:18 2022] PUT /identity/v3/projects/1c4c2ad68a784b19a339e87595635ae1/users/805dfdb1a5d742069c79d48c7d8dab8f/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:19.142490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.159395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe934410-577e-40cc-8853-ee11c0eab48f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.310026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499d065-0186-4eab-8f56-3abe9385b801 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.312381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.313017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.313302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.313973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.314238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.316496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.326994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.327938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.328238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.328519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.329187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.333834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.343116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 950/1891] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:19.348194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.360760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 942/1892] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:19.367788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.369307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0063da-105e-437c-91d4-ede96d51c5bb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.401304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.402078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.402285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.402567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] PATH_INFO: `/v3/projects/5afae4aad81a4e7a890b2159c7ef79b8/users/cec982d610414355bf4e678ec4b6f587/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.404317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329e51f-e742-4961-b7b2-b50a0daa0e43 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.410552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babae43-59e6-412a-a86a-7b6509588843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.411283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babae43-59e6-412a-a86a-7b6509588843 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.411571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babae43-59e6-412a-a86a-7b6509588843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.411852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babae43-59e6-412a-a86a-7b6509588843 admin admin] PATH_INFO: `/v3/projects/1c4c2ad68a784b19a339e87595635ae1/users/805dfdb1a5d742069c79d48c7d8dab8f/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.414500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babae43-59e6-412a-a86a-7b6509588843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4c2ad68a784b19a339e87595635ae1, user_id=805dfdb1a5d742069c79d48c7d8dab8f, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.417380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babae43-59e6-412a-a86a-7b6509588843 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.435577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 943/1893] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:19 2022] PUT /identity/v3/projects/1c4c2ad68a784b19a339e87595635ae1/users/805dfdb1a5d742069c79d48c7d8dab8f/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:19.446161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026377-ed48-453a-899e-d66bf309784f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.446529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026377-ed48-453a-899e-d66bf309784f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.446810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026377-ed48-453a-899e-d66bf309784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.451976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afae4aad81a4e7a890b2159c7ef79b8, user_id=cec982d610414355bf4e678ec4b6f587, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.455222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.483984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c8026377-ed48-453a-899e-d66bf309784f None None] MFA Rules not processed for user `805dfdb1a5d742069c79d48c7d8dab8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:19.485403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c8026377-ed48-453a-899e-d66bf309784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:19.555245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 951/1894] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:19 2022] PUT /identity/v3/projects/5afae4aad81a4e7a890b2159c7ef79b8/users/cec982d610414355bf4e678ec4b6f587/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:19.568277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.601128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 944/1895] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:19.610367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026377-ed48-453a-899e-d66bf309784f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.612929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c8026377-ed48-453a-899e-d66bf309784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:19.676659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d16d9-3451-438e-ad2a-9036eb31dfaf admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.683710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.684328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.684730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.685026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.685716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.687469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.700374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026377-ed48-453a-899e-d66bf309784f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.703246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 952/1896] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:19.705058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:19.715521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4899f-334c-4b7c-b25a-c00c564dab16 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:19.721827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c91ed-e22c-4494-b02d-f18668509c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.726072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c91ed-e22c-4494-b02d-f18668509c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.726072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c91ed-e22c-4494-b02d-f18668509c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.726072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c91ed-e22c-4494-b02d-f18668509c33 admin admin] PATH_INFO: `/v3/projects/5afae4aad81a4e7a890b2159c7ef79b8/users/cec982d610414355bf4e678ec4b6f587/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.726072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c91ed-e22c-4494-b02d-f18668509c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afae4aad81a4e7a890b2159c7ef79b8, user_id=cec982d610414355bf4e678ec4b6f587, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:19.728673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c91ed-e22c-4494-b02d-f18668509c33 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:19.743826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 953/1897] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:19 2022] PUT /identity/v3/projects/5afae4aad81a4e7a890b2159c7ef79b8/users/cec982d610414355bf4e678ec4b6f587/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:19.758542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.758542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.758542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.795863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] MFA Rules not processed for user `cec982d610414355bf4e678ec4b6f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:19.805816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:19.881666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:19.882180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:19.882456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:19.882740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:19.975600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 954/1898] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:19.993164 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:20.008494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:20.010163 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:20.022868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 945/1899] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:20.085715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcb7d9-6ae6-4233-af75-1caff2c1ff8c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:20.085715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af026c8-15ea-44d9-8d09-4406ebb17bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:20.085715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af026c8-15ea-44d9-8d09-4406ebb17bff service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:20.085715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af026c8-15ea-44d9-8d09-4406ebb17bff service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:20.086057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af026c8-15ea-44d9-8d09-4406ebb17bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:20.172401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af026c8-15ea-44d9-8d09-4406ebb17bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:20.174323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af026c8-15ea-44d9-8d09-4406ebb17bff service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:20.180967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:20.193798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5431f-ca39-4aba-b255-8ab8af974757 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:20.204545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:20.205728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:20.206068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:20.206651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:20.214263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 955/1900] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:20.286334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:20.286334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:20.300269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 946/1901] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:20.326562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:20.855763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:20.990362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:20.990362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:20.990812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:21.053319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] MFA Rules not processed for user `cb64ac86a6bc4699ba9e65934efa056f`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:21.054960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:21.192248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 956/1902] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:21.199011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:21.296074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed197eb-127b-4733-84bd-a2db63a462ce service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:21.302995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:21.303880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:21.304380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:21.305420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:21.306362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:21.309304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:21.509417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 947/1903] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:21 2022] POST /identity/v3/projects => generated 373 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:21.509733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:21.525842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74acb1-a412-470b-98df-7a3fdf4ceced None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:21.638624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:21.639434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:21.639792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:21.640139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:21.640949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:21.643706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:21.758011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 957/1904] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:21 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:21.758445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:21.766896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f0735-9ef4-4c97-ad6a-df364869311f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:21.772196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:21.772811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:21.773094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:21.773367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:21.774009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:21.775730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:21.787340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 948/1905] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:21.791819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:21.802642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48f0ba-b788-42ac-b7da-1e70cceefe94 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:21.807821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:21.808597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:21.808882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:21.809158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] PATH_INFO: `/v3/projects/9faf6e036fe946b4879f13ce30671eae/users/82115d01e2c5449c871ca29048fa96dc/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:21.824411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9faf6e036fe946b4879f13ce30671eae, user_id=82115d01e2c5449c871ca29048fa96dc, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:21.827274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:21.924629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 958/1906] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:21 2022] PUT /identity/v3/projects/9faf6e036fe946b4879f13ce30671eae/users/82115d01e2c5449c871ca29048fa96dc/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:21.928732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:21.938508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b8c6b-2f8d-4cfb-8d1e-04754de0933a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:21.993457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:21.994071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:21.994358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:21.994631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:21.995316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:21.996907 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:22.008042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 949/1907] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:22.012284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:22.023126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deffe-2b7b-4526-9843-eb4f95583221 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:22.028108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:22.033625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:22.033625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:22.033625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] PATH_INFO: `/v3/projects/9faf6e036fe946b4879f13ce30671eae/users/82115d01e2c5449c871ca29048fa96dc/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:22.033625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9faf6e036fe946b4879f13ce30671eae, user_id=82115d01e2c5449c871ca29048fa96dc, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:22.034286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:22.044553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 959/1908] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:22 2022] PUT /identity/v3/projects/9faf6e036fe946b4879f13ce30671eae/users/82115d01e2c5449c871ca29048fa96dc/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:22.554748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:22.625841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba1e1-6b81-4efd-ba07-61ec72b16d4c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:22.681715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483974d0-3675-47ee-8829-4b0040e11aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:22.682650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483974d0-3675-47ee-8829-4b0040e11aae service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:22.683136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483974d0-3675-47ee-8829-4b0040e11aae service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:22.683510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483974d0-3675-47ee-8829-4b0040e11aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:22.784555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483974d0-3675-47ee-8829-4b0040e11aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:22.786090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483974d0-3675-47ee-8829-4b0040e11aae service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:22.828993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 950/1909] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:23.433579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:23.435293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:23.508718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499eefa-a475-4368-983a-2c0c239b54b8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:23.513460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:23.514259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:23.514611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:23.514951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] PATH_INFO: `/v3/users/46b4e6260d064b2abfb471e06cd4686a` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:23.516219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b4e6260d064b2abfb471e06cd4686a)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:23.518856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:23.547861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:23.548537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:23.549914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:23.598076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] MFA Rules not processed for user `82115d01e2c5449c871ca29048fa96dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:23.600134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:23.719577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 951/1910] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:23.727551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:23.760275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b4e6260d064b2abfb471e06cd4686a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:23.874307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b4e6260d064b2abfb471e06cd4686a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:23.874307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b4e6260d064b2abfb471e06cd4686a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:23.874307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b4e6260d064b2abfb471e06cd4686a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:23.881721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:23.889794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8d463bf7-447c-41bd-8f3f-795b7c40d0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b4e6260d064b2abfb471e06cd4686a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:23.894618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5fd72-4f7f-456b-a03a-a6504499820d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:23.899766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:23.900651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:23.901301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:23.901672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:23.904267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:23.922740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 960/1911] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:23 2022] DELETE /identity/v3/users/46b4e6260d064b2abfb471e06cd4686a => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:23.929206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:23.929507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:23.929957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:23.953261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:23.954510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:23.988741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:23.990512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:24.008866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 952/1912] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:24.026602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 961/1913] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:24.037658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:24.038950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:24.131281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349208c-c910-4cc4-b982-8eeeb88f72c1 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:24.304096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:24.305269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:24.305693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:24.306190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:24.316740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:24.318799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:24.329775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 953/1914] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:24.401703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:24.830654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:24.831195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:24.844088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9931c-c357-4d05-ace7-824b19aeb256 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:24.851123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:24.852030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:24.852315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:24.852587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] PATH_INFO: `/v3/projects/f120db82790e4eda957d951a7d269269` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:24.853737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f120db82790e4eda957d951a7d269269)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:24.855943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:25.042284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:25.046774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:25.070448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f120db82790e4eda957d951a7d269269', 'request_id': 'req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.077493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f120db82790e4eda957d951a7d269269', 'request_id': 'req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.105895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e1015-8ea6-4cef-85d5-207fba000baf service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:25.159108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f120db82790e4eda957d951a7d269269', 'request_id': 'req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.172669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] The token cache is being invalidate because project f120db82790e4eda957d951a7d269269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:25.173682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.174892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.179405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:25.180151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:25.183321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 962/1915] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:24 2022] DELETE /identity/v3/projects/f120db82790e4eda957d951a7d269269 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:25.183465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:25.183465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:25.184534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:25.190576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:25.200000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:25.274101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bae1e-6f0d-4fb2-bc95-a13f5d7b4855 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:25.279082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:25.280013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:25.280298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:25.280574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] PATH_INFO: `/v3/users/a0ba67173af4465e8090e129323ff795` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:25.281715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ba67173af4465e8090e129323ff795)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:25.283703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:25.287510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:25.289028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:25.301021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 954/1916] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:25.513372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ba67173af4465e8090e129323ff795'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.607023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ba67173af4465e8090e129323ff795'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.607670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ba67173af4465e8090e129323ff795'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.608283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ba67173af4465e8090e129323ff795'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.632493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ba67173af4465e8090e129323ff795'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:25.666477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 963/1917] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:25 2022] DELETE /identity/v3/users/a0ba67173af4465e8090e129323ff795 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:26.105672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:26.107085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.119392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ed07-77aa-40d1-81c2-cade19f14a61 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:26.169106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:26.169722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.170003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.170278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.172123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.197667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:26.199301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.243823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:26.245315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:26.259370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 955/1918] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:26.260353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:26.268666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.278664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f94609-b1e9-47e1-969f-a0ae922a1eb1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:26.288138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f81344-51e5-4240-b698-e366c948c0a2 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:26.289610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:26.291318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.291798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.292235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] PATH_INFO: `/v3/users/3e909b879d674ecf9297b968c9a4d988` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.293301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:26.293820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e909b879d674ecf9297b968c9a4d988)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:26.297836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.297836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.297836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] PATH_INFO: `/v3/projects/1767da9d5c944adc897c682a88bd0f30` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.297836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1767da9d5c944adc897c682a88bd0f30)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:26.298147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:26.298232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:26.521250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e909b879d674ecf9297b968c9a4d988'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.530029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1767da9d5c944adc897c682a88bd0f30', 'request_id': 'req-d5b6380c-7f06-45e0-a238-3e50c46394b7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.544511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1767da9d5c944adc897c682a88bd0f30', 'request_id': 'req-d5b6380c-7f06-45e0-a238-3e50c46394b7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.590826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e909b879d674ecf9297b968c9a4d988'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.593031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e909b879d674ecf9297b968c9a4d988'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.593928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e909b879d674ecf9297b968c9a4d988'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.603474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6142debe-c49d-4a6d-9a89-7c2df52b849a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e909b879d674ecf9297b968c9a4d988'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.635855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 964/1919] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:26 2022] DELETE /identity/v3/users/3e909b879d674ecf9297b968c9a4d988 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:26.640482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.640770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.641043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.661349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:26.662592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.708097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1767da9d5c944adc897c682a88bd0f30', 'request_id': 'req-d5b6380c-7f06-45e0-a238-3e50c46394b7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.731978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] The token cache is being invalidate because project 1767da9d5c944adc897c682a88bd0f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:26.732947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.733616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d5b6380c-7f06-45e0-a238-3e50c46394b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:26.739307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 956/1920] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:26 2022] DELETE /identity/v3/projects/1767da9d5c944adc897c682a88bd0f30 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:26.741683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d13686-ed63-4e28-892d-46db0a654476 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.742278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d13686-ed63-4e28-892d-46db0a654476 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.742565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d13686-ed63-4e28-892d-46db0a654476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.743508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 965/1921] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:26.754418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:26.760578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.791595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-14d13686-ed63-4e28-892d-46db0a654476 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:26.794557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-14d13686-ed63-4e28-892d-46db0a654476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.806305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 957/1922] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:26.816396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.816689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.816971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.837939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:26.860754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:26.862795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.874041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d34fe-a7a7-449d-b0d4-54d96bcaa735 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:26.878641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:26.879545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.892056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.892056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.892056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:26.892351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 958/1923] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:26.892351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:26.900217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:26.901934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:26.914237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 966/1924] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:26.920763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d8a4-70e1-4289-ac81-884cdef07ca6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:26.922004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:26.922693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.922979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.923298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.924056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:26.924371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:26.926175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:26.940330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 959/1925] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:26.945012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:26.956762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d27a5-e875-499c-b948-c2345b5f6ee4 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:26.965629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:26.966589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:26.967094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:26.967604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:26.968477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:26.971697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.085995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 967/1926] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:26 2022] POST /identity/v3/projects => generated 409 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:27.087469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.098514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8004e-93b9-43e8-a303-3e2f30735c2e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:27.172726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:27.173368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.174339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.174684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.175502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:27.177673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.306094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 960/1927] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:27 2022] POST /identity/v3/users => generated 305 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:27.312133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.324675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:27.331459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570f3bb-8554-4d34-99bd-4e663e482689 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:27.337935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:27.338946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.339458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.340194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.342104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:27.345160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.359438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 968/1928] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:27.363568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.378888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5564f-750f-4fc0-b320-6ea470283f46 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:27.408457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:27.409095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.409378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.409665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.410368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:27.419875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.424633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 961/1929] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:27.428779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.443483 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3364cf9-02da-4e35-9c5d-fd60bdc40ef4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:27.444920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbee02-174f-41be-852a-6f98d5ebfc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:27.449513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbee02-174f-41be-852a-6f98d5ebfc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.449513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbee02-174f-41be-852a-6f98d5ebfc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.449513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbee02-174f-41be-852a-6f98d5ebfc29 admin admin] PATH_INFO: `/v3/projects/0965bee3831e47c9a052a6a26b9c0156/users/234392414e4b4ac1a1115583bb8eb503/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.461822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbee02-174f-41be-852a-6f98d5ebfc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0965bee3831e47c9a052a6a26b9c0156, user_id=234392414e4b4ac1a1115583bb8eb503, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:27.464546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbee02-174f-41be-852a-6f98d5ebfc29 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.556420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.558360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:27.573901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca8c2f-fcc0-4e30-993f-b7c38d72db5c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:27.580696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:27.581469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.581756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.582029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] PATH_INFO: `/v3/projects/dbe7e9190ae24a5695c4c108b88d394d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.583192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe7e9190ae24a5695c4c108b88d394d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:27.585179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.607911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 969/1930] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:27 2022] PUT /identity/v3/projects/0965bee3831e47c9a052a6a26b9c0156/users/234392414e4b4ac1a1115583bb8eb503/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:27.618804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.619098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.619411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.647059 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] MFA Rules not processed for user `234392414e4b4ac1a1115583bb8eb503`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:27.648242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:27.724489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 970/1931] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:27.729715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.742551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe7e9190ae24a5695c4c108b88d394d', 'request_id': 'req-a0ecac79-282a-4b13-a72f-1fbce352546a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:27.753216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe7e9190ae24a5695c4c108b88d394d', 'request_id': 'req-a0ecac79-282a-4b13-a72f-1fbce352546a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:27.817752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd55208-1d15-406a-86d2-99afbaa36ca8 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:27.844241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe7e9190ae24a5695c4c108b88d394d', 'request_id': 'req-a0ecac79-282a-4b13-a72f-1fbce352546a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:27.870763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] The token cache is being invalidate because project dbe7e9190ae24a5695c4c108b88d394d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:27.871088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:27.871836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:27.874287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 962/1932] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:27 2022] DELETE /identity/v3/projects/dbe7e9190ae24a5695c4c108b88d394d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:27.881964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:27.884651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:27.922219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:27.923206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:27.924644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:27.926382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:27.927323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:27.930566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:27.997758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecac79-282a-4b13-a72f-1fbce352546a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.006064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.006805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.007091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.007407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] PATH_INFO: `/v3/users/801173c9a43f40a8821d954946f90760` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.008574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=801173c9a43f40a8821d954946f90760)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.010606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.037684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 971/1933] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:27 2022] POST /identity/v3/projects => generated 403 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:28.042267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:28.043929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:28.113203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019e2c-c1b5-410e-9573-108a37710892 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.119297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.120152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.120633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.120973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.121866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.123961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.206563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 972/1934] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:28 2022] POST /identity/v3/users => generated 308 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:28.207725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:28.218574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801173c9a43f40a8821d954946f90760'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:28.226186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903653e-0f82-4be1-a627-3ad82c47176f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.230886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.231531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.231807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.232528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.233196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.234813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.245600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 973/1935] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:28.249856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:28.259717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f67e1-8e77-48dd-99da-52be8bac9c8f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.264456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.265116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.265403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.265682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] PATH_INFO: `/v3/projects/eb0d35cbf9fb4b18b53d95fa97d6f241/users/29134a36ddaf4e99b130f78844639375/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.280921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0d35cbf9fb4b18b53d95fa97d6f241, user_id=29134a36ddaf4e99b130f78844639375, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.284157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.294296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801173c9a43f40a8821d954946f90760'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:28.295794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801173c9a43f40a8821d954946f90760'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:28.297670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801173c9a43f40a8821d954946f90760'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:28.321031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801173c9a43f40a8821d954946f90760'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:28.341823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 974/1936] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:28 2022] PUT /identity/v3/projects/eb0d35cbf9fb4b18b53d95fa97d6f241/users/29134a36ddaf4e99b130f78844639375/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:28.347119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:28.349651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:28.357580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 963/1937] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:27 2022] DELETE /identity/v3/users/801173c9a43f40a8821d954946f90760 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:28.449740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7b5f5-2e4c-4437-8fd9-ee4930f2f12f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.532299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8c8ea-e530-455d-9c23-4a6aaf686219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.532993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8c8ea-e530-455d-9c23-4a6aaf686219 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.533287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8c8ea-e530-455d-9c23-4a6aaf686219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.533566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8c8ea-e530-455d-9c23-4a6aaf686219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.534231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8c8ea-e530-455d-9c23-4a6aaf686219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.535877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8c8ea-e530-455d-9c23-4a6aaf686219 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.564329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 975/1938] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:28.579304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:28.593854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3841b-771f-4418-92e8-a4fa3195cf10 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.600392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.601132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.601431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.601677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] PATH_INFO: `/v3/projects/eb0d35cbf9fb4b18b53d95fa97d6f241/users/29134a36ddaf4e99b130f78844639375/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.604225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0d35cbf9fb4b18b53d95fa97d6f241, user_id=29134a36ddaf4e99b130f78844639375, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.608085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.635883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.636503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.636956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.638921 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 964/1939] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:28 2022] PUT /identity/v3/projects/eb0d35cbf9fb4b18b53d95fa97d6f241/users/29134a36ddaf4e99b130f78844639375/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:28.717005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] MFA Rules not processed for user `29134a36ddaf4e99b130f78844639375`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:28.717005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:28.814897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 976/1940] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:28.833845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:28.837328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:28.899040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474ae1c-2735-4854-ba48-7d0518b103a6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:28.902743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:28.903393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:28.903739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:28.904085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:28.914399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:28.915896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:28.933325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 965/1941] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:28 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:29.361726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:29.361726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:29.370835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:29.372505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:29.372642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc229a5-8fab-4a1e-a3f7-d68c04d40033 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:29.376385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:29.377745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:29.378024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:29.378295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:29.380357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:29.475547 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:29.477460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:29.488018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 977/1942] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:29 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:29.508873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:29.513447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d4098-39b2-470c-9997-c0416ea38ff0 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:29.520441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:29.521226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:29.521608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:29.521952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] PATH_INFO: `/v3/projects/bfd6ef0a30e94211a74cf478f7675592` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:29.523313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd6ef0a30e94211a74cf478f7675592)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:29.525340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:29.668114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd6ef0a30e94211a74cf478f7675592', 'request_id': 'req-9e5597d3-124b-4454-b0f1-eea9cc7a49be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:29.675479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd6ef0a30e94211a74cf478f7675592', 'request_id': 'req-9e5597d3-124b-4454-b0f1-eea9cc7a49be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:29.754793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd6ef0a30e94211a74cf478f7675592', 'request_id': 'req-9e5597d3-124b-4454-b0f1-eea9cc7a49be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:29.766274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] The token cache is being invalidate because project bfd6ef0a30e94211a74cf478f7675592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:29.766572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:29.767287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:29.769387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 966/1943] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:29 2022] DELETE /identity/v3/projects/bfd6ef0a30e94211a74cf478f7675592 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:29.773521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:29.784891 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7be3c-ddb8-463e-9086-5a720660cf1d service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:29.842297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:29.842939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:29.843249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:29.843524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] PATH_INFO: `/v3/users/805dfdb1a5d742069c79d48c7d8dab8f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:29.844661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805dfdb1a5d742069c79d48c7d8dab8f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:29.848255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:29.926844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:30.094211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805dfdb1a5d742069c79d48c7d8dab8f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.155419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805dfdb1a5d742069c79d48c7d8dab8f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.156086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805dfdb1a5d742069c79d48c7d8dab8f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.156735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805dfdb1a5d742069c79d48c7d8dab8f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.164492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805dfdb1a5d742069c79d48c7d8dab8f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.196709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 978/1944] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:29 2022] DELETE /identity/v3/users/805dfdb1a5d742069c79d48c7d8dab8f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:30.472468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:30.474488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:30.492214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5597d3-124b-4454-b0f1-eea9cc7a49be admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:30.605838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:30.606516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:30.606799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:30.607268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] PATH_INFO: `/v3/users/daf1691634ae4be6ae5c6e2e127c3154` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:30.610068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf1691634ae4be6ae5c6e2e127c3154)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:30.624810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:30.757143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:30.758761 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:30.784920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a984d-67a0-412e-951a-779d6b865701 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:30.802598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:30.803518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:30.803811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:30.804091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] PATH_INFO: `/v3/projects/1c4c2ad68a784b19a339e87595635ae1` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:30.805453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4c2ad68a784b19a339e87595635ae1)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:30.808016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:30.915918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf1691634ae4be6ae5c6e2e127c3154'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.969793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4c2ad68a784b19a339e87595635ae1', 'request_id': 'req-310e7a30-841f-48da-b574-35d724c58674'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:30.977624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4c2ad68a784b19a339e87595635ae1', 'request_id': 'req-310e7a30-841f-48da-b574-35d724c58674'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.062307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf1691634ae4be6ae5c6e2e127c3154'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.063547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf1691634ae4be6ae5c6e2e127c3154'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.065130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf1691634ae4be6ae5c6e2e127c3154'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.074635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df1ca4f6-e62f-453b-a9a1-c72dbc2f7b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf1691634ae4be6ae5c6e2e127c3154'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.107938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 967/1945] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:30 2022] DELETE /identity/v3/users/daf1691634ae4be6ae5c6e2e127c3154 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:31.111602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.116652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.116652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.134908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:31.136696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:31.160916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4c2ad68a784b19a339e87595635ae1', 'request_id': 'req-310e7a30-841f-48da-b574-35d724c58674'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.180171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] The token cache is being invalidate because project 1c4c2ad68a784b19a339e87595635ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:31.180484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.182484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-310e7a30-841f-48da-b574-35d724c58674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:31.184439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 979/1946] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:30 2022] DELETE /identity/v3/projects/1c4c2ad68a784b19a339e87595635ae1 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:31.191466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2845b-e147-4a1b-b762-44c9b9418e73 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.192404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2845b-e147-4a1b-b762-44c9b9418e73 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.193433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2845b-e147-4a1b-b762-44c9b9418e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.228068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 968/1947] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:31.242244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7df2845b-e147-4a1b-b762-44c9b9418e73 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:31.243866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7df2845b-e147-4a1b-b762-44c9b9418e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:31.253525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.256326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:31.258892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 980/1948] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:31.266745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.267893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.268403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.301517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:31.306115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:31.325535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 981/1949] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:31.333932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.358076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b621ae6-ac24-49c5-a743-056b26d154c0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.368680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.369529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.372064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.375306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.375306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:31.375306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.378696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:31.384628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0a4b0-3532-4dfb-a2b8-3c7766863ba6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.399920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.403858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 982/1950] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:31.417228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.417318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.417318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.417318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.418691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.418885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8af12-9803-472b-9460-685db6f99152 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.420585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:31.424270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.424882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.425214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.425507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.426275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.428386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:31.434155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 969/1951] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:31.534060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 983/1952] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:31 2022] POST /identity/v3/projects => generated 359 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:31.535765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.545665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15611a2f-3f51-4904-af8f-f2fa68841175 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.655841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.656470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.656758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.657548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.658291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.660535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:31.762143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 970/1953] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:31 2022] POST /identity/v3/users => generated 280 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:31.762296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.776605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e81b7b-4113-4ae8-b187-44ab6909703e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.790479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.791365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.791657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.791937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.793241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.794871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:31.807822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 984/1954] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:31.815677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.829648 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278337-2e1b-408d-b2c6-ae5855c9f7ae admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.835462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e46746-39ae-47f2-b252-78ff7602ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.836210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e46746-39ae-47f2-b252-78ff7602ac85 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.836598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e46746-39ae-47f2-b252-78ff7602ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.836989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e46746-39ae-47f2-b252-78ff7602ac85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.837803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e46746-39ae-47f2-b252-78ff7602ac85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.839561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e46746-39ae-47f2-b252-78ff7602ac85 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:31.853840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 971/1955] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:31.858041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:31.876750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe502c07-7896-473f-8453-2409d32abf5b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:31.888601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:31.889903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:31.890396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:31.890893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] PATH_INFO: `/v3/projects/5d89928db4e942878b0f718854d95429/users/8bde57f9a13c4795890667852faa96ee/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:31.910292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d89928db4e942878b0f718854d95429, user_id=8bde57f9a13c4795890667852faa96ee, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:31.914301 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:32.033925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 985/1956] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:31 2022] PUT /identity/v3/projects/5d89928db4e942878b0f718854d95429/users/8bde57f9a13c4795890667852faa96ee/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:32.046515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:32.046866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:32.047137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:32.078134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] MFA Rules not processed for user `8bde57f9a13c4795890667852faa96ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:32.080820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:32.157606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:32.167767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 972/1957] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:32.328829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:32.330831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:32.426705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e639-318e-4b8f-86ce-0ce987b8a870 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:32.433711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:32.434030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:32.434266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:32.435526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:32.448287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:32.450063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:32.460892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 986/1958] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:32.482656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:32.502242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:32.561152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:32.562640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:32.564804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ce394-f999-4eab-bdcd-9db648542bc8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:32.571677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:32.572624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:32.573137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:32.573560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] PATH_INFO: `/v3/users/8bde57f9a13c4795890667852faa96ee` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:32.575079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bde57f9a13c4795890667852faa96ee)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:32.577303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:32.581955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4925dec-65cb-4175-a88c-e896b00b5075 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:32.596805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:32.597873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:32.598358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:32.599017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] PATH_INFO: `/v3/projects/b598156982eb437b9abbe7c3516dec91` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:32.600704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b598156982eb437b9abbe7c3516dec91)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:32.602982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:32.760624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bde57f9a13c4795890667852faa96ee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.765691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b598156982eb437b9abbe7c3516dec91', 'request_id': 'req-0504e7ef-4886-4bde-9984-19da45f5d5a2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.775642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b598156982eb437b9abbe7c3516dec91', 'request_id': 'req-0504e7ef-4886-4bde-9984-19da45f5d5a2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.864605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bde57f9a13c4795890667852faa96ee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.867601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bde57f9a13c4795890667852faa96ee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.869115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bde57f9a13c4795890667852faa96ee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.878438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10685e72-4ed3-41c5-a250-4820bb810c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bde57f9a13c4795890667852faa96ee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.903886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 973/1959] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:32 2022] DELETE /identity/v3/users/8bde57f9a13c4795890667852faa96ee => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:32.909376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:32.909740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:32.910082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:32.921072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b598156982eb437b9abbe7c3516dec91', 'request_id': 'req-0504e7ef-4886-4bde-9984-19da45f5d5a2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.933124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:32.934423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:32.944265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] The token cache is being invalidate because project b598156982eb437b9abbe7c3516dec91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:32.944881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.946098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:32.950724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 987/1960] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:32 2022] DELETE /identity/v3/projects/b598156982eb437b9abbe7c3516dec91 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:32.956192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:32.957888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:33.019338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 974/1961] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:33.027363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:33.028839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:33.037348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504e7ef-4886-4bde-9984-19da45f5d5a2 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:33.052690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:33.053773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:33.054275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:33.054732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] PATH_INFO: `/v3/users/4524e23f01cf465d9996a0d359958331` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:33.056155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4524e23f01cf465d9996a0d359958331)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:33.058097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:33.097862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f304-0c54-43c3-aa9d-b44d409ccfee None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:33.102008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:33.102632 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:33.103014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:33.103318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:33.113668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:33.115851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:33.125456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 975/1962] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:33.229962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4524e23f01cf465d9996a0d359958331'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.327102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4524e23f01cf465d9996a0d359958331'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.327315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4524e23f01cf465d9996a0d359958331'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.327956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4524e23f01cf465d9996a0d359958331'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.337448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4524e23f01cf465d9996a0d359958331'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.379918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 988/1963] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:32 2022] DELETE /identity/v3/users/4524e23f01cf465d9996a0d359958331 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:33.632490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:33.634002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:33.648615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec05333-00a4-4f33-97a5-fcc25ea77447 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:33.734857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:33.735557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:33.739383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:33.739383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] PATH_INFO: `/v3/projects/5d89928db4e942878b0f718854d95429` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:33.739383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d89928db4e942878b0f718854d95429)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:33.739673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:33.740196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:33.935266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d89928db4e942878b0f718854d95429', 'request_id': 'req-d0617d82-c467-4d0b-96a7-8f0998c236b1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.937363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d89928db4e942878b0f718854d95429', 'request_id': 'req-d0617d82-c467-4d0b-96a7-8f0998c236b1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:33.963019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:33.964650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:33.982334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0984c-5736-4396-bf60-b53b3a97698a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.043393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.044300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.044687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.044936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] PATH_INFO: `/v3/projects/25a77bb4e1f541ca9e74fceaa6403fba` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.047841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a77bb4e1f541ca9e74fceaa6403fba)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.050909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.052633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d89928db4e942878b0f718854d95429', 'request_id': 'req-d0617d82-c467-4d0b-96a7-8f0998c236b1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.067211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] The token cache is being invalidate because project 5d89928db4e942878b0f718854d95429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:34.067686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.068423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d0617d82-c467-4d0b-96a7-8f0998c236b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.074119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 976/1964] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:33 2022] DELETE /identity/v3/projects/5d89928db4e942878b0f718854d95429 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:34.075615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0d2c-5cef-40de-b444-7232f28d2f08 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.076519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0d2c-5cef-40de-b444-7232f28d2f08 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.076808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0d2c-5cef-40de-b444-7232f28d2f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.098817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0aac0d2c-5cef-40de-b444-7232f28d2f08 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:34.100087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0aac0d2c-5cef-40de-b444-7232f28d2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:34.110115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 977/1965] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:34.117790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.118076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.118350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.138323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:34.139500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:34.150219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 978/1966] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:34.154233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.166600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49732eb-96bd-48fc-9ed6-724dd747bceb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.168688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:34.175369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.176102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.176448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.176766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.177423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:34.177760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.179546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.194847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a77bb4e1f541ca9e74fceaa6403fba', 'request_id': 'req-3563bd53-fca7-4535-be8b-bf6b7862b162'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.200255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 979/1967] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:16:34.202547 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a77bb4e1f541ca9e74fceaa6403fba', 'request_id': 'req-3563bd53-fca7-4535-be8b-bf6b7862b162'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.204178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.221884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0316be-ff38-44d4-8dbe-47a093a1e001 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.278422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.279088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.279426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.280011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.280644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.282546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.303210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a77bb4e1f541ca9e74fceaa6403fba', 'request_id': 'req-3563bd53-fca7-4535-be8b-bf6b7862b162'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.314573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] The token cache is being invalidate because project 25a77bb4e1f541ca9e74fceaa6403fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:34.315046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.315718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.317217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 989/1968] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:33 2022] DELETE /identity/v3/projects/25a77bb4e1f541ca9e74fceaa6403fba => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:34.321055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.322425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:34.336017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563bd53-fca7-4535-be8b-bf6b7862b162 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.341958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.341958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.341958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.341958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] PATH_INFO: `/v3/users/cec982d610414355bf4e678ec4b6f587` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.343236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec982d610414355bf4e678ec4b6f587)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.345193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.397241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 980/1969] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:34 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:34.401109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.402644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:34.475522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44a39-d92b-4bfd-afb2-1b8922164fc4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.475522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.475522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.475522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.475904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.475904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.477075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.546670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec982d610414355bf4e678ec4b6f587'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.552498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 981/1970] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:34 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:34.553525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.563362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264798c-5960-43e4-8e3f-50f88b7f67d9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.568701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.569303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.569588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.569865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.570554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.572157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.581941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 982/1971] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:34.585672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.594941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf308cac-824c-426a-8439-78488ca73d45 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.600132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.600707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.601109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.601357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.602030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.603624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.614515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 983/1972] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:34.618533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.628791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec982d610414355bf4e678ec4b6f587'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.628941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2131d-74d0-4ff1-a1b1-efbd1792cb1c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.629486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec982d610414355bf4e678ec4b6f587'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.630127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec982d610414355bf4e678ec4b6f587'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.633702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.639095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.639095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.639095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] PATH_INFO: `/v3/projects/e1eaf4013736444396e2e047f91f4542/users/9fd1b338198a4f82bce89910156300ce/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.639384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c93027d-8530-4157-9d25-b623d1721ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec982d610414355bf4e678ec4b6f587'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:34.649311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1eaf4013736444396e2e047f91f4542, user_id=9fd1b338198a4f82bce89910156300ce, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.651938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.655428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 990/1973] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:34 2022] DELETE /identity/v3/users/cec982d610414355bf4e678ec4b6f587 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:34.749358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 984/1974] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:34 2022] PUT /identity/v3/projects/e1eaf4013736444396e2e047f91f4542/users/9fd1b338198a4f82bce89910156300ce/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:34.761002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.761359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.761638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.790720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] MFA Rules not processed for user `9fd1b338198a4f82bce89910156300ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:34.792376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:34.865514 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 991/1975] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:34.873324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:34.875084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:34.949503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29662-d0fe-492c-8834-162c24d366c7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:34.954251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f95339-10b9-4478-9007-3f74ebfdf97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:34.954905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95339-10b9-4478-9007-3f74ebfdf97b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:34.955232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95339-10b9-4478-9007-3f74ebfdf97b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:34.955651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95339-10b9-4478-9007-3f74ebfdf97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:34.958681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:34.966160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f95339-10b9-4478-9007-3f74ebfdf97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:34.968407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f95339-10b9-4478-9007-3f74ebfdf97b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:34.977791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 985/1976] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:35.103287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:35.105132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:35.177711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f02b5-3804-4d37-965c-c19625a78241 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:35.182269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:35.182907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.183218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.183497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] PATH_INFO: `/v3/projects/5afae4aad81a4e7a890b2159c7ef79b8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.184629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afae4aad81a4e7a890b2159c7ef79b8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:35.186600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:35.320022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afae4aad81a4e7a890b2159c7ef79b8', 'request_id': 'req-a5e08057-16a3-4787-a78c-6ce1a0df71df'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:35.328609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afae4aad81a4e7a890b2159c7ef79b8', 'request_id': 'req-a5e08057-16a3-4787-a78c-6ce1a0df71df'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:35.415491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afae4aad81a4e7a890b2159c7ef79b8', 'request_id': 'req-a5e08057-16a3-4787-a78c-6ce1a0df71df'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:35.436704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] The token cache is being invalidate because project 5afae4aad81a4e7a890b2159c7ef79b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:35.436950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:35.437619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a5e08057-16a3-4787-a78c-6ce1a0df71df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:35.439290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 992/1977] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:35 2022] DELETE /identity/v3/projects/5afae4aad81a4e7a890b2159c7ef79b8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:35.444399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.444897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.445250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.465380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:35.466622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:35.533374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 986/1978] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:35.540057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.540347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.540625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.561874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:35.563733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:35.574139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 993/1979] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:35.578440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:35.588610 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57623d1-55fc-4350-ba97-dccf62d50b58 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:35.593264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:35.593862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.594148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.594421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.595066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:35.595420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:35.596993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:35.608281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 987/1980] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:35.612559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:35.621807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8219c-cd8a-4c11-ae85-a504eba0b5fc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:35.627364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:35.628014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.628292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.628560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.629219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:35.631094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:35.744844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 994/1981] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:35 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:35.749215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:35.759590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7b09-f733-4e96-baf8-0524dc647e73 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:35.791701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:35.823354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:35.823867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.824144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.824414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.825042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:35.827022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:35.955084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 988/1982] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:35 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:35.959315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:35.969659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18634ef1-6dc6-42ed-8908-691737ef4851 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:35.974583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:35.975316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:35.975981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:35.975981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:35.976802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:35.978594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:35.989974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 995/1983] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:35.994203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:36.005142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e80c1-3d35-4433-89f1-750a4b3b2ab9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:36.009852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53798a15-79de-4953-93d3-b798417d6528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:36.010461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53798a15-79de-4953-93d3-b798417d6528 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:36.010747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53798a15-79de-4953-93d3-b798417d6528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:36.011022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53798a15-79de-4953-93d3-b798417d6528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:36.011697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53798a15-79de-4953-93d3-b798417d6528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:36.013294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53798a15-79de-4953-93d3-b798417d6528 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:36.026160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 989/1984] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:36.027962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:36.038570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5fce8-edb1-4a21-a00b-d4ff554182d8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:36.050726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:36.051430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:36.051717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:36.051996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] PATH_INFO: `/v3/projects/723137b6e4764ec6b58b46f84a05a37e/users/6c886285823a45a0b921a180d5bf4f1e/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:36.068289 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=723137b6e4764ec6b58b46f84a05a37e, user_id=6c886285823a45a0b921a180d5bf4f1e, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:36.071108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:36.155904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 996/1985] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:36 2022] PUT /identity/v3/projects/723137b6e4764ec6b58b46f84a05a37e/users/6c886285823a45a0b921a180d5bf4f1e/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:36.161051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34eb9d-c56a-45be-ae05-27545774dd50 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:36.161338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34eb9d-c56a-45be-ae05-27545774dd50 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:36.161611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34eb9d-c56a-45be-ae05-27545774dd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:36.182864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4d34eb9d-c56a-45be-ae05-27545774dd50 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:36.184165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4d34eb9d-c56a-45be-ae05-27545774dd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:36.243386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 990/1986] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:36.253297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:36.255102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:36.311909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd50630-29aa-4440-9505-c8c5e0748838 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:36.315603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:36.316278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:36.316560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:36.316837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:36.325949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:36.327489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:36.337096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 997/1987] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:36.773526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:37.773933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e59e5-2a72-4656-b206-b57a0422a055 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:37.774480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e59e5-2a72-4656-b206-b57a0422a055 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:37.774770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e59e5-2a72-4656-b206-b57a0422a055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:37.828363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0e5e59e5-2a72-4656-b206-b57a0422a055 None None] MFA Rules not processed for user `6c886285823a45a0b921a180d5bf4f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:37.830400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0e5e59e5-2a72-4656-b206-b57a0422a055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:37.887602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:37.899235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 991/1988] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:37.910181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:37.911856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:37.969202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c3acd-7b0e-4578-bde4-2acece384471 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:37.972939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:37.973537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:37.973815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:37.974082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:37.982371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:37.983825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:37.994539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 998/1989] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:38.509585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:39.417388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:39.531816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:39.966080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:40.132333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:40.478587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:40.594733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:40.972178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:41.095744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:42.015376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614172a-c8a7-4925-bd1d-c5f51a28f193 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:42.016053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614172a-c8a7-4925-bd1d-c5f51a28f193 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:42.016053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614172a-c8a7-4925-bd1d-c5f51a28f193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:42.036343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f614172a-c8a7-4925-bd1d-c5f51a28f193 None None] MFA Rules not processed for user `6c886285823a45a0b921a180d5bf4f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:42.037657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f614172a-c8a7-4925-bd1d-c5f51a28f193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:42.044653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 992/1990] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:42.145286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:42.331121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:43.215710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:43.503823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:44.097754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:44.508981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:44.632641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:44.703427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.028816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.097245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.125778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.154276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.297592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.331811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.415426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.495371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.549740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.880367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:45.935155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:46.315481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:46.400700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:46.735848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:46.903337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:47.091970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:47.093691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:47.108153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a6887-93e0-4885-83d3-10a942e2bdfb service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:47.113386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:47.114013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:47.114298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:47.114577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] PATH_INFO: `/v3/users/234392414e4b4ac1a1115583bb8eb503` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:47.115767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=234392414e4b4ac1a1115583bb8eb503)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:47.117933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:47.351553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234392414e4b4ac1a1115583bb8eb503'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:47.428643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234392414e4b4ac1a1115583bb8eb503'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:47.429214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234392414e4b4ac1a1115583bb8eb503'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:47.429832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234392414e4b4ac1a1115583bb8eb503'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:47.437287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234392414e4b4ac1a1115583bb8eb503'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:47.455397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 999/1991] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:47 2022] DELETE /identity/v3/users/234392414e4b4ac1a1115583bb8eb503 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:47.459788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:47.460301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:47.460825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:47.464803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:47.482368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:47.483657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:47.547120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 993/1992] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:47.559668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:47.561092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:47.621945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a704e-71a3-4238-9923-a1e5441b11b0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:47.626829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:47.628445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:47.631508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:47.631508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:47.631508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:47.639964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:47.641464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:47.652283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1000/1993] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:48.120635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:48.234446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:48.656280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:48.658283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:48.671944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51342c92-dae5-4e34-8000-6604f09880b3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:48.674019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:48.695365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:48.696320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:48.696601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:48.696872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] PATH_INFO: `/v3/projects/0965bee3831e47c9a052a6a26b9c0156` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:48.698083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0965bee3831e47c9a052a6a26b9c0156)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:48.700045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:48.831203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0965bee3831e47c9a052a6a26b9c0156', 'request_id': 'req-6cca7af2-1464-4c51-ba08-db118234e2c4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:48.843930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0965bee3831e47c9a052a6a26b9c0156', 'request_id': 'req-6cca7af2-1464-4c51-ba08-db118234e2c4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:48.924902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0965bee3831e47c9a052a6a26b9c0156', 'request_id': 'req-6cca7af2-1464-4c51-ba08-db118234e2c4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:48.936837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] The token cache is being invalidate because project 0965bee3831e47c9a052a6a26b9c0156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:48.937380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:48.940349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:48.943407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 994/1994] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:48 2022] DELETE /identity/v3/projects/0965bee3831e47c9a052a6a26b9c0156 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:48.946659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:48.974512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b68a-3c50-44d2-863e-03c47e3735df service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:49.102528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:49.103212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:49.103587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:49.103771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] PATH_INFO: `/v3/users/29134a36ddaf4e99b130f78844639375` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:49.105034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29134a36ddaf4e99b130f78844639375)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:49.107265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:49.362219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29134a36ddaf4e99b130f78844639375'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:49.375561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:49.454663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29134a36ddaf4e99b130f78844639375'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:49.454663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29134a36ddaf4e99b130f78844639375'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:49.454663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29134a36ddaf4e99b130f78844639375'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:49.464936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0206bd27-de8e-4a17-b853-7ffa11ab6c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29134a36ddaf4e99b130f78844639375'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:49.488624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1001/1995] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:48 2022] DELETE /identity/v3/users/29134a36ddaf4e99b130f78844639375 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:49.623430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:50.193910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:50.195751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:50.236672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7af2-1464-4c51-ba08-db118234e2c4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:50.301901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:50.302555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:50.302820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:50.303090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] PATH_INFO: `/v3/projects/eb0d35cbf9fb4b18b53d95fa97d6f241` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:50.304238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0d35cbf9fb4b18b53d95fa97d6f241)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:50.306168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:50.421819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0d35cbf9fb4b18b53d95fa97d6f241', 'request_id': 'req-ff55443b-66ae-4fba-8737-19ba4b443972'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:50.428847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0d35cbf9fb4b18b53d95fa97d6f241', 'request_id': 'req-ff55443b-66ae-4fba-8737-19ba4b443972'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:50.559773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0d35cbf9fb4b18b53d95fa97d6f241', 'request_id': 'req-ff55443b-66ae-4fba-8737-19ba4b443972'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:50.571152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] The token cache is being invalidate because project eb0d35cbf9fb4b18b53d95fa97d6f241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:50.571536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:50.572187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff55443b-66ae-4fba-8737-19ba4b443972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:50.573749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 995/1996] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:50 2022] DELETE /identity/v3/projects/eb0d35cbf9fb4b18b53d95fa97d6f241 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:50.578724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:50.579242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:50.579358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:50.612338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:50.613853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:50.689385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1002/1997] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:50.699709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:50.699709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:50.699709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:50.721486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:50.723029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:50.731768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 996/1998] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:50.736437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:50.749518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0551b-8f99-442c-8952-c6db1e8c6338 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:50.755642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:50.757798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:50.757798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:50.757977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:50.758875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:50.759261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:50.760900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:50.773553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1003/1999] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:50.778287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:50.788799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361cefd-4558-48d9-8b8f-bb2308235a20 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:50.794125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:50.795038 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:50.795502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:50.795773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:50.796824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:50.799217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:50.807049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:51.008015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 997/2000] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:50 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:51.012312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:51.029981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50bb81-390e-461b-82ba-6253f9399395 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:51.140056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:51.140686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.141120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.141977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.142735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:51.144823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:51.268087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1004/2001] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:51 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:51.272307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:51.284372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45a636-2efe-4118-b2fd-4f402ade51b0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:51.289683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:51.290494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.290782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.291053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.291743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:51.293638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:51.304970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 998/2002] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:51.308957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:51.320304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019b7f3-2b27-4ef4-bd6e-ad8760001314 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:51.326593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1efaa-ee2c-4b0f-91c7-23edf2d69ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:51.327250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1efaa-ee2c-4b0f-91c7-23edf2d69ead admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.327647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1efaa-ee2c-4b0f-91c7-23edf2d69ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.327825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1efaa-ee2c-4b0f-91c7-23edf2d69ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.328476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1efaa-ee2c-4b0f-91c7-23edf2d69ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:51.330149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1efaa-ee2c-4b0f-91c7-23edf2d69ead admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:51.341027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1005/2003] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:51.349224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:51.361307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff85b6-12c1-4f0d-8f33-9251f495497b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:51.366238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:51.366809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.367120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.367383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] PATH_INFO: `/v3/projects/317d10395c254ee2922daadf807d9585/users/93eba635fb98498c89b2a6a7d3cc3377/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.388037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] RBAC: Authorizing `identity:create_grant(project_id=317d10395c254ee2922daadf807d9585, user_id=93eba635fb98498c89b2a6a7d3cc3377, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:51.390683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:51.480624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 999/2004] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:51 2022] PUT /identity/v3/projects/317d10395c254ee2922daadf807d9585/users/93eba635fb98498c89b2a6a7d3cc3377/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:51.494795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.495087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.495425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.526657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] MFA Rules not processed for user `93eba635fb98498c89b2a6a7d3cc3377`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:51.527956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:51.598297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1006/2005] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:51.603059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:51.626411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:51.686539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d6f7d-08cc-4218-b031-97da5347228c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:51.692819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:51.692819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.692819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.692819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.693454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:51.695586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:51.768014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1000/2006] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:51 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:51.773438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:51.803982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8174c2a-6cfd-4684-ba06-5098315952ff None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:51.896016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:51.896721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:51.897082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:51.897500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:51.898532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:51.901812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.015642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1007/2007] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:51 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:52.016960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:52.027369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f85ff-2d4d-4f62-9606-0d4abcbc818f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:52.032880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:52.033499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.033779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.034051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.034690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:52.036347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.050903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1001/2008] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:52.058839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:52.069185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3d67b-46ab-48c9-bc7c-9207c66d3c92 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:52.073894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98501401-0b5f-479f-984e-1da28d05cfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:52.074554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501401-0b5f-479f-984e-1da28d05cfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.074834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501401-0b5f-479f-984e-1da28d05cfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.075103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501401-0b5f-479f-984e-1da28d05cfa8 admin admin] PATH_INFO: `/v3/projects/fc8ee43f01e94959b03626bc2eb1b41b/users/036ba808d5c24893ad1e8463281dac3c/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.090128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98501401-0b5f-479f-984e-1da28d05cfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8ee43f01e94959b03626bc2eb1b41b, user_id=036ba808d5c24893ad1e8463281dac3c, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:52.092889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98501401-0b5f-479f-984e-1da28d05cfa8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.119909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:52.148588 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1008/2009] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:52 2022] PUT /identity/v3/projects/fc8ee43f01e94959b03626bc2eb1b41b/users/036ba808d5c24893ad1e8463281dac3c/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:52.152782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:52.164532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c769f03-8fce-4548-949a-04e923ee0b08 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:52.227553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:52.228326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.231595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.231977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.233234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:52.235068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.249235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1002/2010] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:52.258464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378b7b4-b9ff-44ab-925a-6981de47a60f None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.258950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378b7b4-b9ff-44ab-925a-6981de47a60f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.259385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378b7b4-b9ff-44ab-925a-6981de47a60f None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.261082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1009/2011] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:16:52.265736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:52.277449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c8f32-04e0-4c19-a72f-d95d1fcaaf95 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:52.283258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fe5eb-f2c5-430b-bd8e-c42ca637bee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:52.284004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fe5eb-f2c5-430b-bd8e-c42ca637bee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.284445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fe5eb-f2c5-430b-bd8e-c42ca637bee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.284887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fe5eb-f2c5-430b-bd8e-c42ca637bee9 admin admin] PATH_INFO: `/v3/projects/fc8ee43f01e94959b03626bc2eb1b41b/users/036ba808d5c24893ad1e8463281dac3c/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.288265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.288544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.288814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.292046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fe5eb-f2c5-430b-bd8e-c42ca637bee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8ee43f01e94959b03626bc2eb1b41b, user_id=036ba808d5c24893ad1e8463281dac3c, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:52.296836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fe5eb-f2c5-430b-bd8e-c42ca637bee9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.309217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1003/2012] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:52 2022] PUT /identity/v3/projects/fc8ee43f01e94959b03626bc2eb1b41b/users/036ba808d5c24893ad1e8463281dac3c/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:52.310449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] MFA Rules not processed for user `751cbd1e77824b83aa3c9c03c637fadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:52.312621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:52.320418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.320893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.321332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.354316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] MFA Rules not processed for user `036ba808d5c24893ad1e8463281dac3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:52.355584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:52.405202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1010/2013] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:52.418021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:52.419905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:52.437253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cfabc-3a62-45bc-9bca-d0311a37f5ac None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:52.444985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1004/2014] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:52.448810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:52.449353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.458476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.458476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.462910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:52.464803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.466175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:52.467650 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:52.475114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:52.478297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1011/2015] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:52.538400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215bea-2eeb-40a5-a7bd-cd7540634eb7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:52.540672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:52.541286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:52.541569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:52.541849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:52.551343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:52.552954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:52.562572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1005/2016] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:16:53.246078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:53.247722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:53.342172 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cb456-769b-4fe4-95ce-67558fb832bf service placement] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:53.347634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:53.348264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:53.348641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:53.348880 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] PATH_INFO: `/v3/users/cb64ac86a6bc4699ba9e65934efa056f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:53.350010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb64ac86a6bc4699ba9e65934efa056f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:53.351973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:53.459838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:53.624821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb64ac86a6bc4699ba9e65934efa056f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:53.689470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb64ac86a6bc4699ba9e65934efa056f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:53.690092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb64ac86a6bc4699ba9e65934efa056f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:53.690697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb64ac86a6bc4699ba9e65934efa056f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:53.698068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb64ac86a6bc4699ba9e65934efa056f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:53.716018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1012/2017] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:53 2022] DELETE /identity/v3/users/cb64ac86a6bc4699ba9e65934efa056f => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:54.022236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:54.033688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899904e1-97a2-4e87-b881-6af8de0e85a7 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:54.035961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:54.037618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:54.095310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:54.095949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:54.096239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:54.096514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] PATH_INFO: `/v3/projects/bd469d812fad4dd18a0c73980f756e8d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:54.097649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd469d812fad4dd18a0c73980f756e8d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:54.099624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:54.111361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6f1c-7184-4401-b1d6-343057e45d19 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:54.116106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:54.116693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:54.116976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:54.117251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] PATH_INFO: `/v3/users/93eba635fb98498c89b2a6a7d3cc3377` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:54.118397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93eba635fb98498c89b2a6a7d3cc3377)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:54.120378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:54.191738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd469d812fad4dd18a0c73980f756e8d', 'request_id': 'req-4e28488a-746d-41f2-a609-61d62d8f4140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.197835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd469d812fad4dd18a0c73980f756e8d', 'request_id': 'req-4e28488a-746d-41f2-a609-61d62d8f4140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.245346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd469d812fad4dd18a0c73980f756e8d', 'request_id': 'req-4e28488a-746d-41f2-a609-61d62d8f4140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.256527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] The token cache is being invalidate because project bd469d812fad4dd18a0c73980f756e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:54.256817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.257454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.258960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1006/2018] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:54 2022] DELETE /identity/v3/projects/bd469d812fad4dd18a0c73980f756e8d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:54.259211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93eba635fb98498c89b2a6a7d3cc3377'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.263141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:54.264627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:54.304965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:54.319618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93eba635fb98498c89b2a6a7d3cc3377'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.320355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93eba635fb98498c89b2a6a7d3cc3377'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.320968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93eba635fb98498c89b2a6a7d3cc3377'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.328689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c171474b-d432-4167-a6c9-7dad56eb5484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93eba635fb98498c89b2a6a7d3cc3377'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.329666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28488a-746d-41f2-a609-61d62d8f4140 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:54.334726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:54.335486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:54.335859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:54.336189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] PATH_INFO: `/v3/users/82115d01e2c5449c871ca29048fa96dc` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:54.337435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82115d01e2c5449c871ca29048fa96dc)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:54.339394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:54.347239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1013/2019] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:54 2022] DELETE /identity/v3/users/93eba635fb98498c89b2a6a7d3cc3377 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:54.352051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:54.352195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:54.352375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:54.372321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:54.373124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:54.432990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1014/2020] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:54.448148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:54.449465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:54.510553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2311b-abe9-49c6-af21-53ab62fe7464 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:54.514772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:54.515545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:54.515879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:54.516319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:54.523335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82115d01e2c5449c871ca29048fa96dc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.527795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:54.530002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:54.537872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:54.544781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1015/2021] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:54.603217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82115d01e2c5449c871ca29048fa96dc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.603882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82115d01e2c5449c871ca29048fa96dc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.604519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82115d01e2c5449c871ca29048fa96dc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.613227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b564ca53-8c61-48f0-bbcf-8038e3e7fff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82115d01e2c5449c871ca29048fa96dc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:54.631915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1007/2022] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:54 2022] DELETE /identity/v3/users/82115d01e2c5449c871ca29048fa96dc => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:54.916550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:54.918043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:54.934925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7429-c690-461e-9e21-44b33960128c service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.017857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.018500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.018780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.019054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] PATH_INFO: `/v3/projects/9faf6e036fe946b4879f13ce30671eae` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.020248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9faf6e036fe946b4879f13ce30671eae)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.022208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.144952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9faf6e036fe946b4879f13ce30671eae', 'request_id': 'req-00571ac0-09b0-4568-83f0-bd1056e9761b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.151624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9faf6e036fe946b4879f13ce30671eae', 'request_id': 'req-00571ac0-09b0-4568-83f0-bd1056e9761b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.224337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9faf6e036fe946b4879f13ce30671eae', 'request_id': 'req-00571ac0-09b0-4568-83f0-bd1056e9761b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.237526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] The token cache is being invalidate because project 9faf6e036fe946b4879f13ce30671eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:55.237934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.238651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.240196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1016/2023] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:54 2022] DELETE /identity/v3/projects/9faf6e036fe946b4879f13ce30671eae => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:55.245211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41b83-2433-4047-b0a3-f7f0f92ff747 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.245691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41b83-2433-4047-b0a3-f7f0f92ff747 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.245782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41b83-2433-4047-b0a3-f7f0f92ff747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.265931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-69f41b83-2433-4047-b0a3-f7f0f92ff747 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:55.267858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69f41b83-2433-4047-b0a3-f7f0f92ff747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:55.313564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.316844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:55.337845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1008/2024] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:55.339444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.339913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.340646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.374962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:55.376994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:55.386954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1009/2025] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:55.391987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.402551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7fcd2-0178-49e6-a4ee-306a1b89ffcc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.407395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:55.409847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.409847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.409847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.409847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.410445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:55.410879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.410988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571ac0-09b0-4568-83f0-bd1056e9761b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.412535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.415735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.416370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.416652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.416952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] PATH_INFO: `/v3/projects/317d10395c254ee2922daadf807d9585` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.418158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] RBAC: Authorizing `identity:delete_project(project_id=317d10395c254ee2922daadf807d9585)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.420850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.425018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1010/2026] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:16:55.429270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.439704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254aac5-47fc-46a4-bd62-f615b98cfa89 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.444516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.445119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.445398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.445676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.446301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.448337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.559026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317d10395c254ee2922daadf807d9585', 'request_id': 'req-cf363ffc-159d-4974-bac1-234c4939815d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.565819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317d10395c254ee2922daadf807d9585', 'request_id': 'req-cf363ffc-159d-4974-bac1-234c4939815d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.594176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1011/2027] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:55 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:55.598287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.611748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:55.657319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317d10395c254ee2922daadf807d9585', 'request_id': 'req-cf363ffc-159d-4974-bac1-234c4939815d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.660180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891139d-44b9-4fe6-bc02-70b8cedac396 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.666108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.666810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.667090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.667397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.668058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.669983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.670093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] The token cache is being invalidate because project 317d10395c254ee2922daadf807d9585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:55.670392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.671069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:55.673323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1017/2028] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:55 2022] DELETE /identity/v3/projects/317d10395c254ee2922daadf807d9585 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:55.678177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.679603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:55.761603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1012/2029] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:55 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:55.766363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf363ffc-159d-4974-bac1-234c4939815d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.768264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.772248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:55.777723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.778400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.778688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.778973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] PATH_INFO: `/v3/users/036ba808d5c24893ad1e8463281dac3c` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.780490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=036ba808d5c24893ad1e8463281dac3c)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.782976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.794228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407241d1-717b-495f-91a0-784fce095dbd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.798995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.801587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.801587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.801587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.801587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.804628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.828724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1013/2030] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:55.832507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.846992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91279f27-905e-42cf-b5de-4ec0ea7592b0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.852860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.853884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.854271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.854890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.855493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.857230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:55.874505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1014/2031] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:55.878740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:55.890851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee709a2-47f6-4771-86bf-0b64ba8271c6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:55.893739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec414a89-dff9-4ade-805f-b3d0559440ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:55.894511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec414a89-dff9-4ade-805f-b3d0559440ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:55.894804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec414a89-dff9-4ade-805f-b3d0559440ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:55.895085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec414a89-dff9-4ade-805f-b3d0559440ac admin admin] PATH_INFO: `/v3/projects/79c9b5645b594526b129a790a2d87c23/users/a279ab23939f46419496baa77db69947/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:55.919037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec414a89-dff9-4ade-805f-b3d0559440ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c9b5645b594526b129a790a2d87c23, user_id=a279ab23939f46419496baa77db69947, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:55.922051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec414a89-dff9-4ade-805f-b3d0559440ac admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:56.047292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:56.063696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1015/2032] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:55 2022] PUT /identity/v3/projects/79c9b5645b594526b129a790a2d87c23/users/a279ab23939f46419496baa77db69947/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:56.075432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:56.076060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:56.076348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:56.077670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036ba808d5c24893ad1e8463281dac3c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.111031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] MFA Rules not processed for user `a279ab23939f46419496baa77db69947`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:56.112378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:56.181092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036ba808d5c24893ad1e8463281dac3c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.181998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036ba808d5c24893ad1e8463281dac3c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.182968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036ba808d5c24893ad1e8463281dac3c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.192252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036ba808d5c24893ad1e8463281dac3c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.199588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1016/2033] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:16:56.211841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:56.223442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:56.223649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1018/2034] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:16:55 2022] DELETE /identity/v3/users/036ba808d5c24893ad1e8463281dac3c => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:16:56.294384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98a07e-46e1-4bd1-9d34-81f348c9daf2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:56.358213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56a626-488a-4443-bea7-3de7c4a1b58d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:56.358931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56a626-488a-4443-bea7-3de7c4a1b58d service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:56.359323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56a626-488a-4443-bea7-3de7c4a1b58d service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:56.359669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56a626-488a-4443-bea7-3de7c4a1b58d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:56.371181 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:56.427312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56a626-488a-4443-bea7-3de7c4a1b58d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:56.428903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56a626-488a-4443-bea7-3de7c4a1b58d service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:56.447298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1017/2035] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:16:56.648109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:56.649612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:56.740570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda85d32-952b-4668-8c1b-d8dc061f1650 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:56.745143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:56.745783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:56.746062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:56.746337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] PATH_INFO: `/v3/projects/fc8ee43f01e94959b03626bc2eb1b41b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:56.747490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc8ee43f01e94959b03626bc2eb1b41b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:56.749406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:56.855021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc8ee43f01e94959b03626bc2eb1b41b', 'request_id': 'req-be691dd2-b276-45b2-a12e-25b8d55f0978'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.862900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc8ee43f01e94959b03626bc2eb1b41b', 'request_id': 'req-be691dd2-b276-45b2-a12e-25b8d55f0978'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.914433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc8ee43f01e94959b03626bc2eb1b41b', 'request_id': 'req-be691dd2-b276-45b2-a12e-25b8d55f0978'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.926503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] The token cache is being invalidate because project fc8ee43f01e94959b03626bc2eb1b41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:16:56.926797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.927480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-be691dd2-b276-45b2-a12e-25b8d55f0978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:16:56.929040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1019/2036] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:16:56 2022] DELETE /identity/v3/projects/fc8ee43f01e94959b03626bc2eb1b41b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:56.934012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:56.934308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:56.934619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:56.956741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:56.957977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:57.047817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1018/2037] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:57.052932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.053613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.053613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.099934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:57.101188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:57.130983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1020/2038] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:57.131462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.156199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7063508-3b5c-4302-83e8-0c14407a77d9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.180388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.182223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.182518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.182802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.184675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:16:57.185000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.187483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:57.206302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1019/2039] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:16:57.210898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.222639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f5dc0-618b-4fcb-96e0-301897a61540 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.235593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.235593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.235593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.235593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.236013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.237796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:57.385541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1021/2040] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:57 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:57.389816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.407844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0359-08c9-4b77-b0d7-993736f6be13 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.460110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:57.476487 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.477113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.477396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.477696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.478336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.480359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:57.554476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1020/2041] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:57 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:57.558957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.571140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b6647-6b3b-4703-86ff-bd901f43e962 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.576601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.577288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.577797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.578263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.583058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.583058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:57.584095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:57.593141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1022/2042] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:57.597261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.607799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce05e-ba3a-4c16-88cd-430e1f1bc4e3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.613019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0ceef-f55d-4665-9cd1-2d4e4f0b8dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.613647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0ceef-f55d-4665-9cd1-2d4e4f0b8dce admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.613928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0ceef-f55d-4665-9cd1-2d4e4f0b8dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.614203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0ceef-f55d-4665-9cd1-2d4e4f0b8dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.614856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0ceef-f55d-4665-9cd1-2d4e4f0b8dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.616524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0ceef-f55d-4665-9cd1-2d4e4f0b8dce admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:57.628967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1021/2043] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:57.633111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.643794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ab13-ff11-4a2c-a38f-69994c1ed702 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.648951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.649811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.650586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.650865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] PATH_INFO: `/v3/projects/a8e8240a546f49f7b3fd3f9417ee5127/users/bdb7d4943cd3482d89c0b981f9d7a5ad/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.669652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e8240a546f49f7b3fd3f9417ee5127, user_id=bdb7d4943cd3482d89c0b981f9d7a5ad, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.673287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:57.741940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1023/2044] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:57 2022] PUT /identity/v3/projects/a8e8240a546f49f7b3fd3f9417ee5127/users/bdb7d4943cd3482d89c0b981f9d7a5ad/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:57.752957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.753346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.753690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.782227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] MFA Rules not processed for user `bdb7d4943cd3482d89c0b981f9d7a5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:57.783665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:57.855704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1022/2045] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:57.863636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:57.941972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4b3f-967f-4908-bb6b-7855e7b772b9 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:57.946937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:57.947709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:57.948119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:57.948408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:57.949135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:57.951193 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:58.061364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1024/2046] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:16:57 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:16:58.065994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:58.080320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe68a8a-4610-4ff7-a942-2d512fb58cb5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:58.139749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:58.140373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.140653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.140923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:58.142099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:58.144264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:58.231833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1023/2047] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:16:58 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:16:58.233556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:58.246940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09e24-66b7-406d-ae8c-2028b6fb1d10 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:58.263537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:58.265202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.265202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.265607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:58.266922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:58.268703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:58.290173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1025/2048] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:16:58.295150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:58.305946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4fd3a-9417-47e9-8c55-f11ce9530693 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:58.311881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:58.312587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.313935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.313935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] PATH_INFO: `/v3/projects/053eb93add054de5837c7a8d0f399a03/users/5fa3dca88bc646e59f8c3d0bdb9f62db/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:58.333093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053eb93add054de5837c7a8d0f399a03, user_id=5fa3dca88bc646e59f8c3d0bdb9f62db, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:58.335818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:58.417770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1024/2049] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:58 2022] PUT /identity/v3/projects/053eb93add054de5837c7a8d0f399a03/users/5fa3dca88bc646e59f8c3d0bdb9f62db/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:58.422551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:58.441446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256ebbf-0c74-4ad7-ab27-0029c754d9a4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:58.518049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4cfcc-53b6-4198-a03c-37b37132b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:58.518695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4cfcc-53b6-4198-a03c-37b37132b760 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.518974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4cfcc-53b6-4198-a03c-37b37132b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.519276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4cfcc-53b6-4198-a03c-37b37132b760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:58.519968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4cfcc-53b6-4198-a03c-37b37132b760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:58.521717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4cfcc-53b6-4198-a03c-37b37132b760 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:58.534151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1026/2050] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:16:58.538451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:58.549828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b529e24-6942-4fb8-95f9-b0808e9978d7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:58.554847 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:58.555531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.555818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.556095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] PATH_INFO: `/v3/projects/053eb93add054de5837c7a8d0f399a03/users/5fa3dca88bc646e59f8c3d0bdb9f62db/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:58.558100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053eb93add054de5837c7a8d0f399a03, user_id=5fa3dca88bc646e59f8c3d0bdb9f62db, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:58.560807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:58.591689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1025/2051] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:16:58 2022] PUT /identity/v3/projects/053eb93add054de5837c7a8d0f399a03/users/5fa3dca88bc646e59f8c3d0bdb9f62db/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:16:58.593980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.604699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.604699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:58.630725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] MFA Rules not processed for user `5fa3dca88bc646e59f8c3d0bdb9f62db`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:16:58.632103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:58.670522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:58.742333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1027/2052] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:16:58.793051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:16:58.807969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:16:58.815490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:16:58.981342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53950216-b8de-4fa6-b004-bdfc5ae1a242 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:16:58.985374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:16:58.986063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:16:58.986416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:16:58.986753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:16:59.044655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:16:59.046161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:16:59.056107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1026/2053] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:00.663464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:00.665063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:00.731245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f7f0b-a866-4ef9-b9ce-89bb93586795 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:00.736293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:00.736928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:00.737211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:00.737486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] PATH_INFO: `/v3/users/ca6855123d0740aba728a2379419cc95` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:00.738615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6855123d0740aba728a2379419cc95)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:00.740564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:01.024638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6855123d0740aba728a2379419cc95'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.094782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6855123d0740aba728a2379419cc95'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.095393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6855123d0740aba728a2379419cc95'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.096037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6855123d0740aba728a2379419cc95'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.103516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-25e7ac0a-5144-4b5d-b352-8bd69cb64f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6855123d0740aba728a2379419cc95'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.121294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1028/2054] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:00 2022] DELETE /identity/v3/users/ca6855123d0740aba728a2379419cc95 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:01.361834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:01.372979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29210912-4721-4fc3-8e78-1c9d7cd01ae8 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:01.439199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:01.439828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:01.440107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:01.440376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] PATH_INFO: `/v3/projects/30beae855402476b9e1e542be6cc7e88` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:01.441478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30beae855402476b9e1e542be6cc7e88)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:01.443383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:01.582700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30beae855402476b9e1e542be6cc7e88', 'request_id': 'req-ba7cb123-997a-4d85-ba7f-2a48f2815709'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.588754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30beae855402476b9e1e542be6cc7e88', 'request_id': 'req-ba7cb123-997a-4d85-ba7f-2a48f2815709'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.662350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30beae855402476b9e1e542be6cc7e88', 'request_id': 'req-ba7cb123-997a-4d85-ba7f-2a48f2815709'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.663577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:01.673316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] The token cache is being invalidate because project 30beae855402476b9e1e542be6cc7e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:01.673630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.674266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ba7cb123-997a-4d85-ba7f-2a48f2815709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:01.675788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1027/2055] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:01 2022] DELETE /identity/v3/projects/30beae855402476b9e1e542be6cc7e88 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:01.680819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:01.681274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:01.681361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:01.701011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:01.702220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:01.749284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:01.764576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1029/2056] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:01.772936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:01.773226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:01.773505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:01.793499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:01.794752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:01.802766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1028/2057] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:01.815354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:01.826117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f09e6-e3f0-49c4-8b3f-b425a6506dc7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:01.831685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:01.832299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:01.832823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:01.833095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:01.833740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:01.834043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:01.835688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:01.847406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1030/2058] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:01.853411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:01.862517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b827c2-485a-4074-87f0-aa42eb8b2697 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:01.867929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:01.868578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:01.868857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:01.869728 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:01.870157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:01.872136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:01.963074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1029/2059] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:01 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:01.967318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:01.977651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d278-2476-44d1-af48-b7b9977693cb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:02.040222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:02.040835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:02.041113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:02.041385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:02.042034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:02.044067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:02.131105 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1031/2060] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:01 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:02.132441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:02.143369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900eb92-9f7f-483d-b1dd-c64cb3adc646 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:02.148276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:02.148981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:02.149276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:02.149542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:02.150258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:02.152006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:02.162877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1030/2061] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:02.167158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:02.177937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9bd2b-9b32-40a9-97c2-2ccf4f0abd04 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:02.182664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c329b-7deb-4937-97df-c9f6ae45a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:02.183312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c329b-7deb-4937-97df-c9f6ae45a4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:02.183592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c329b-7deb-4937-97df-c9f6ae45a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:02.183862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c329b-7deb-4937-97df-c9f6ae45a4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:02.184504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c329b-7deb-4937-97df-c9f6ae45a4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:02.186086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c329b-7deb-4937-97df-c9f6ae45a4d6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:02.197534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1032/2062] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:02.202264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:02.213071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c8e91-db3b-4a33-9a3e-e4c7dc9372fc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:02.217856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:02.218523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:02.218799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:02.219296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] PATH_INFO: `/v3/projects/c9aaa16702a5422b90578033420da9bd/users/90f07b101240464387f3fbf92c925ec7/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:02.234357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9aaa16702a5422b90578033420da9bd, user_id=90f07b101240464387f3fbf92c925ec7, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:02.237530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:02.295222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1031/2063] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:02 2022] PUT /identity/v3/projects/c9aaa16702a5422b90578033420da9bd/users/90f07b101240464387f3fbf92c925ec7/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:02.300951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:02.301313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:02.301658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:02.322112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:02.323381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:02.403974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1033/2064] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:02.414526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:02.416424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:02.476658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72388ec7-4f78-4917-8615-e44f99f69069 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:02.480353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e349192-21b1-4140-ac84-42b5c4fb7d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:02.481104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e349192-21b1-4140-ac84-42b5c4fb7d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:02.481250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e349192-21b1-4140-ac84-42b5c4fb7d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:02.481585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e349192-21b1-4140-ac84-42b5c4fb7d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:02.492598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e349192-21b1-4140-ac84-42b5c4fb7d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:02.494113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e349192-21b1-4140-ac84-42b5c4fb7d0f service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:02.504725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1032/2065] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:03.533838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:03.535332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:03.547560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64170235-a9ac-4b3c-81dd-cdca2b558407 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:03.553166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:03.553808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:03.554094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:03.554373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] PATH_INFO: `/v3/users/9fd1b338198a4f82bce89910156300ce` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:03.555565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd1b338198a4f82bce89910156300ce)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:03.557500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:03.768299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd1b338198a4f82bce89910156300ce'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:03.771273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:03.877538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd1b338198a4f82bce89910156300ce'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:03.878261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd1b338198a4f82bce89910156300ce'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:03.878890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd1b338198a4f82bce89910156300ce'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:03.886852 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd1b338198a4f82bce89910156300ce'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:03.907810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1034/2066] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:03 2022] DELETE /identity/v3/users/9fd1b338198a4f82bce89910156300ce => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:03.918917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:03.920615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:03.920981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:03.940841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:03.942150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:04.003749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1033/2067] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:04.018967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:04.018967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:04.105114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b75203-6f29-4bb6-afd4-887061dab741 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:04.109896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:04.110518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:04.114583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:04.114583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:04.121186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:04.122629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:04.133147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1035/2068] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:04.590166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:04.592263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:04.752938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d6440-216f-4ec7-80cc-704fe411cafa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:04.752938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:04.752938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:04.752938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:04.754870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] PATH_INFO: `/v3/users/bdb7d4943cd3482d89c0b981f9d7a5ad` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:04.754870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb7d4943cd3482d89c0b981f9d7a5ad)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:04.754870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:04.754870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:05.058690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.058690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:05.073296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb7d4943cd3482d89c0b981f9d7a5ad'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.081043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d238fd7-7952-4829-80d2-ba65ec43d100 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.090073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.094741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.094741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.094741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] PATH_INFO: `/v3/projects/e1eaf4013736444396e2e047f91f4542` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.094741 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1eaf4013736444396e2e047f91f4542)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.096565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.182528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb7d4943cd3482d89c0b981f9d7a5ad'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.183596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb7d4943cd3482d89c0b981f9d7a5ad'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.184721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb7d4943cd3482d89c0b981f9d7a5ad'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.200494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be76799e-e0d7-422f-ad7f-dc9cd99ab5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb7d4943cd3482d89c0b981f9d7a5ad'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.218474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1034/2069] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:04 2022] DELETE /identity/v3/users/bdb7d4943cd3482d89c0b981f9d7a5ad => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:05.227218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.228228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.232362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.263368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:05.263615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1eaf4013736444396e2e047f91f4542', 'request_id': 'req-4d60f53d-2462-4d7f-b565-ad3929cc695b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.264796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:05.270791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1eaf4013736444396e2e047f91f4542', 'request_id': 'req-4d60f53d-2462-4d7f-b565-ad3929cc695b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.344583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1eaf4013736444396e2e047f91f4542', 'request_id': 'req-4d60f53d-2462-4d7f-b565-ad3929cc695b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.357308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1035/2070] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:05.364358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.365802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:05.366668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] The token cache is being invalidate because project e1eaf4013736444396e2e047f91f4542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:05.367032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.367729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4d60f53d-2462-4d7f-b565-ad3929cc695b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:05.369529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1036/2071] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:05 2022] DELETE /identity/v3/projects/e1eaf4013736444396e2e047f91f4542 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:05.374376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd07f5-8529-459b-ae98-30729b090353 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.374669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd07f5-8529-459b-ae98-30729b090353 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.374951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd07f5-8529-459b-ae98-30729b090353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.403735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-59cd07f5-8529-459b-ae98-30729b090353 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:05.406176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-59cd07f5-8529-459b-ae98-30729b090353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:05.414412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1037/2072] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:05.421746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.423203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.423203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.445445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:05.447635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:05.450273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9467532-80a0-4d03-b51a-7bb25eba002c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.455424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.456637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.457073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.457988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.460452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:05.463069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1038/2073] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:05.469385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.476955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d761fe8-7a38-408f-8e48-c2f606f364ab None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.478064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.479667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.481255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.481921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.482276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.482630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.485324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:05.490536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.490536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.492674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1036/2074] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:05.505264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1039/2075] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:05.509916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.520095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3bc4e-caa9-4eea-9483-47923b8e76ce service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.535410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.536335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.536785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.537673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.538750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.541433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.645243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1037/2076] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:05 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:05.649320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.660401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661f0fe-24a7-4a9e-9e19-2b46480a5868 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.687865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:05.729306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.734478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.734478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.734478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.734478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.734478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.863211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1040/2077] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:05 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:05.864890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.876119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d003c92-de73-477c-aa7c-bf16e136beed admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.882912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.884543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.885037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.885528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.886530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.889323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.903806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1038/2078] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:05.909022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.922303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376994f-6b33-4765-b050-f2fa0eb8e1f6 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.927151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca8ef3-8377-46a5-8ae3-c79e00260da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.928105 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca8ef3-8377-46a5-8ae3-c79e00260da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.928612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca8ef3-8377-46a5-8ae3-c79e00260da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.929114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca8ef3-8377-46a5-8ae3-c79e00260da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:05.929987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca8ef3-8377-46a5-8ae3-c79e00260da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:05.935534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca8ef3-8377-46a5-8ae3-c79e00260da0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:05.947479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1041/2079] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:17:05.951841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:05.969920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96355c72-ae74-47c3-9d72-9ab8f15f0c7e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:05.978703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:05.980173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:05.980745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:05.981223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] PATH_INFO: `/v3/projects/17ee24894ef8439b9a81f8336ce8a0ff/users/cabadbcddd854aa3812f6c0e8989064f/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.002823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ee24894ef8439b9a81f8336ce8a0ff, user_id=cabadbcddd854aa3812f6c0e8989064f, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.019101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.121488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1039/2080] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:05 2022] PUT /identity/v3/projects/17ee24894ef8439b9a81f8336ce8a0ff/users/cabadbcddd854aa3812f6c0e8989064f/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:06.132485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.132954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.133374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.152423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.154596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:06.199630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] MFA Rules not processed for user `cabadbcddd854aa3812f6c0e8989064f`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:06.201016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:06.279039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76665cab-2066-4a7f-8bf6-6d069e56ef07 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.284228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.284934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.285285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.285658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] PATH_INFO: `/v3/projects/a8e8240a546f49f7b3fd3f9417ee5127` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.287068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e8240a546f49f7b3fd3f9417ee5127)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.289078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.297561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1042/2081] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:06.302203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.318107 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee1798-b47c-4093-9282-addef3c11fb1 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.324692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.327372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.327372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.327372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.327372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.329524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.424467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e8240a546f49f7b3fd3f9417ee5127', 'request_id': 'req-80b3e624-3895-48ac-a575-6d86d022773e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:06.434310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e8240a546f49f7b3fd3f9417ee5127', 'request_id': 'req-80b3e624-3895-48ac-a575-6d86d022773e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:06.460660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1043/2082] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:06 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:06.468579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.514219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e8240a546f49f7b3fd3f9417ee5127', 'request_id': 'req-80b3e624-3895-48ac-a575-6d86d022773e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:06.542441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] The token cache is being invalidate because project a8e8240a546f49f7b3fd3f9417ee5127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:06.543066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:06.545705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:06.553696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1040/2083] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:06 2022] DELETE /identity/v3/projects/a8e8240a546f49f7b3fd3f9417ee5127 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:17:06.555051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.558472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:06.611304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6403aa-d27f-4134-bc15-e2466b7b3c4a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.622161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.622756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.623112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.623543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.624866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.628267 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.672930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3e624-3895-48ac-a575-6d86d022773e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.682922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.683862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.684362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.684791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] PATH_INFO: `/v3/users/5fa3dca88bc646e59f8c3d0bdb9f62db` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.686338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa3dca88bc646e59f8c3d0bdb9f62db)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.690741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.717140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1044/2084] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:06 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:06.721412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.723431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:06.741090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a174c7-a0f3-4385-afd3-ff779a6cdf82 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.746041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.746667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.750097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.750097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.750097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.750893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.761985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1045/2085] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:17:06.765673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.776131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef63deb-3e18-4a78-bbfb-1ec80afc3357 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.785459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.786361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.786801 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.787277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.788082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.790824 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.798866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:06.804603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1046/2086] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:17:06.808596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.821552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7d67e-8beb-41c1-9ece-29f0f3c31adc admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:06.827830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:06.829004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:06.829529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:06.829923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] PATH_INFO: `/v3/projects/1d14d87770a34979939c7978384d0f67/users/47f799a34fd84eb8b36cfe196f174520/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:06.860441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d14d87770a34979939c7978384d0f67, user_id=47f799a34fd84eb8b36cfe196f174520, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:06.864321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:06.928588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa3dca88bc646e59f8c3d0bdb9f62db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:06.953229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1047/2087] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:06 2022] PUT /identity/v3/projects/1d14d87770a34979939c7978384d0f67/users/47f799a34fd84eb8b36cfe196f174520/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:06.963119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:06.965152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:07.005960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa3dca88bc646e59f8c3d0bdb9f62db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.006730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa3dca88bc646e59f8c3d0bdb9f62db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.007643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa3dca88bc646e59f8c3d0bdb9f62db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.019700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-461ec53b-7188-47a3-8e01-a2def0b37036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa3dca88bc646e59f8c3d0bdb9f62db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.053296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1041/2088] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:06 2022] DELETE /identity/v3/users/5fa3dca88bc646e59f8c3d0bdb9f62db => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:07.064936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.069706 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.071495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.074086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba8413-c131-4264-8987-e32ac15fa187 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:07.149097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] MFA Rules not processed for user `47f799a34fd84eb8b36cfe196f174520`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:07.150684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:07.151535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.152007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.152202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:07.152859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] PATH_INFO: `/v3/users/a279ab23939f46419496baa77db69947` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.154272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a279ab23939f46419496baa77db69947)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:07.158155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:07.302656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1042/2089] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:07.307211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:07.308773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:07.344368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625bd4-e8f1-401e-89a8-c31ffd05c9dd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:07.348837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:07.349736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.350312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.350773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.351808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:07.353922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:07.364772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a279ab23939f46419496baa77db69947'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.449646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a279ab23939f46419496baa77db69947'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.451666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a279ab23939f46419496baa77db69947'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.455185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a279ab23939f46419496baa77db69947'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.466585 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1043/2090] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:07 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:07.467816 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:07.470911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c335124-a089-4451-b9e1-51f2528397a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a279ab23939f46419496baa77db69947'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:07.473013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:07.510705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1048/2091] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:06 2022] DELETE /identity/v3/users/a279ab23939f46419496baa77db69947 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:07.515470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.515470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.516372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.565239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:07.566464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:07.621879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b359-f083-4500-9fa2-b5692fac47fc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:07.687925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1049/2092] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:07.706189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:07.711531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:07.736642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:07.737306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.737592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.737866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.739525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:07.740818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:07.790989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd83a2-329c-46c5-9a6b-9f90b2de17fc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:07.796263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:07.797387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.797387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.797610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.810488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:07.812082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:07.824037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1050/2093] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:07.883545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1044/2094] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:07 2022] POST /identity/v3/users => generated 295 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:07.884775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:07.894840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a14100-1b41-4d65-ba31-27515c971021 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:07.900843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:07.907734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.907734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.907734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.907734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:07.907734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:07.915196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1051/2095] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:07.919604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:07.933026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c228ac-103d-453e-a632-28e856da9bd2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:07.938800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:07.939444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:07.940786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:07.940786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] PATH_INFO: `/v3/projects/ee404cbdccb04607993a4d310a36378b/users/82bf7775c3e14531afeb30740de507c6/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:07.963138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee404cbdccb04607993a4d310a36378b, user_id=82bf7775c3e14531afeb30740de507c6, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:07.966782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:07.971774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:07.986068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:07.988192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.007097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d7c7a-de03-4f24-b655-d82ccbeccadc admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:08.025151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:08.026251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.026650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.027184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] PATH_INFO: `/v3/projects/053eb93add054de5837c7a8d0f399a03` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.028643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053eb93add054de5837c7a8d0f399a03)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:08.034181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:08.069340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1045/2096] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:07 2022] PUT /identity/v3/projects/ee404cbdccb04607993a4d310a36378b/users/82bf7775c3e14531afeb30740de507c6/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:08.070892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:08.161566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf9eb-bd8b-4df3-9b3c-12a34a562946 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:08.168723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:08.169587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.170266 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.176631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.176631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:08.176631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:08.193639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1046/2097] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:08.203504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:08.215434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053eb93add054de5837c7a8d0f399a03', 'request_id': 'req-a679af1b-4a70-4ff3-a86e-d878867900c5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.227444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437607e6-1480-4005-8ca5-1c63838096fd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:08.229410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053eb93add054de5837c7a8d0f399a03', 'request_id': 'req-a679af1b-4a70-4ff3-a86e-d878867900c5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.323807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053eb93add054de5837c7a8d0f399a03', 'request_id': 'req-a679af1b-4a70-4ff3-a86e-d878867900c5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.328329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51280136-adbe-4fcc-b111-02b384fee393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:08.329220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51280136-adbe-4fcc-b111-02b384fee393 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.329662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51280136-adbe-4fcc-b111-02b384fee393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.330167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51280136-adbe-4fcc-b111-02b384fee393 admin admin] PATH_INFO: `/v3/projects/ee404cbdccb04607993a4d310a36378b/users/82bf7775c3e14531afeb30740de507c6/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.332384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51280136-adbe-4fcc-b111-02b384fee393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee404cbdccb04607993a4d310a36378b, user_id=82bf7775c3e14531afeb30740de507c6, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:08.335802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51280136-adbe-4fcc-b111-02b384fee393 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:08.350705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1047/2098] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:08 2022] PUT /identity/v3/projects/ee404cbdccb04607993a4d310a36378b/users/82bf7775c3e14531afeb30740de507c6/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:08.353258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] The token cache is being invalidate because project 053eb93add054de5837c7a8d0f399a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:08.356499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.358488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a679af1b-4a70-4ff3-a86e-d878867900c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.361064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751fea2-d038-4959-b2f5-9af96bdf9f80 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.361846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751fea2-d038-4959-b2f5-9af96bdf9f80 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.362185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751fea2-d038-4959-b2f5-9af96bdf9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.364396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1052/2099] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:07 2022] DELETE /identity/v3/projects/053eb93add054de5837c7a8d0f399a03 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:08.365772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.366062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.366341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.403747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:08.405263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.409046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a751fea2-d038-4959-b2f5-9af96bdf9f80 None None] MFA Rules not processed for user `82bf7775c3e14531afeb30740de507c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:08.410538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a751fea2-d038-4959-b2f5-9af96bdf9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.547967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1053/2100] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:08.547967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:08.547967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.564644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5524b-68b0-4f13-beac-5c03656a4005 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:08.574003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:08.575354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.575672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.575951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] PATH_INFO: `/v3/projects/79c9b5645b594526b129a790a2d87c23` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.577483 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c9b5645b594526b129a790a2d87c23)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:08.580043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:08.581576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1048/2101] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:08.584249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.586504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.589913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.637679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:08.639061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.657219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1049/2102] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:08.659119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:08.662343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.746818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26d6e7-fa1e-4995-9160-fcf85114ff64 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:08.752239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:08.753347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.753798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.754232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:08.756658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:08.793625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c9b5645b594526b129a790a2d87c23', 'request_id': 'req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.816018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c9b5645b594526b129a790a2d87c23', 'request_id': 'req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.862378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:08.864108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:08.889486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1050/2103] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:08.891395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:08.892269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c9b5645b594526b129a790a2d87c23', 'request_id': 'req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.910211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d274cd-0074-4da8-bb65-10958dca3103 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:08.913070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] The token cache is being invalidate because project 79c9b5645b594526b129a790a2d87c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:08.913365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.914446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf2a4c4b-cdd3-4c0b-be90-7347a4f6e5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:08.916558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1054/2104] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:08 2022] DELETE /identity/v3/projects/79c9b5645b594526b129a790a2d87c23 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:08.924338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:08.943857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70acd-c0ce-4e9a-8b54-9293839657f2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:08.947386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70acd-c0ce-4e9a-8b54-9293839657f2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:08.949106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70acd-c0ce-4e9a-8b54-9293839657f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.002139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6ed70acd-c0ce-4e9a-8b54-9293839657f2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:09.006632 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6ed70acd-c0ce-4e9a-8b54-9293839657f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:09.018890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.019575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.020017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.020482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.021197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:09.021462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.023157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.036282 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1055/2105] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:09.043213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1051/2106] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:09.044603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.044902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.045182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.047727 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.051252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:09.077754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:09.077754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:09.087919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db021a5-3f32-4390-8d32-26e591708516 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.090114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1056/2107] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:09.091490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.107082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bb3cc-5681-49d0-ada8-2d0d2c08906c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.110469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.110469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.110469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.110469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.110898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.116957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.118823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.119945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.120228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.120511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.122242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:09.123019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.124333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.154824 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1057/2108] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:09.160942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.201453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234147-079f-497e-9adf-d9a368940626 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.217215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.217938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.218227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.218502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.219875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.230502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.268297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:09.325478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1052/2109] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/projects => generated 375 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:09.330408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.352216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1058/2110] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:09.356252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.431319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139809a-63c0-49ca-abaa-d42bfb3a9421 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.432228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d884f7-3dac-4ee9-a723-d8cace2de3ca admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.436096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b74b8-076e-43ba-817c-f87f5cfc70cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.436716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b74b8-076e-43ba-817c-f87f5cfc70cd admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.436994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b74b8-076e-43ba-817c-f87f5cfc70cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.437106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.437280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b74b8-076e-43ba-817c-f87f5cfc70cd admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.438248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.438537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.438640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b74b8-076e-43ba-817c-f87f5cfc70cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.438812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.439481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.440588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b74b8-076e-43ba-817c-f87f5cfc70cd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.441924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.549178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1053/2111] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:09.549178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.549178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.549178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.577604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1059/2112] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:09.579116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.588804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] MFA Rules not processed for user `90f07b101240464387f3fbf92c925ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:09.589894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:09.591817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232f8c-11c6-42cc-821b-ea2efa9c5dff admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.598591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.603831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.603831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.603831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.603831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.603831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.617227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1060/2113] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:09.618434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.630445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf8b95-b4e8-48dd-8b2e-1aacb950e9d0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.636111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.636489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.636836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.637099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.637788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.639742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.653647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1061/2114] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:09.653647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.664438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd446-d7b9-4cf5-8169-5fad1d568b3a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.671358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1054/2115] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:09.671358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.675651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.676311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.676589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.676965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.677717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.680843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.691768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8f724-bd7c-4391-97cf-c8379462e10f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.696518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.697043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1062/2116] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:09.697130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.697399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.697724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.698353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.698462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.700129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.702356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:09.711240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1055/2117] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:09.712186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.722501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164cfcc-e4b6-4eb3-9d60-412fd7d7a623 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.732034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15731dce-38cb-4f76-a5b0-022318e66f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.732786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15731dce-38cb-4f76-a5b0-022318e66f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.733284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15731dce-38cb-4f76-a5b0-022318e66f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.733638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15731dce-38cb-4f76-a5b0-022318e66f90 admin admin] PATH_INFO: `/v3/projects/e9655193b10a47e6ba071e70d0d8b7aa/users/981d5bb01b35421ba8afe0df824ba5f6/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.764822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15731dce-38cb-4f76-a5b0-022318e66f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9655193b10a47e6ba071e70d0d8b7aa, user_id=981d5bb01b35421ba8afe0df824ba5f6, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.778480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15731dce-38cb-4f76-a5b0-022318e66f90 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.822731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515053ef-bac0-47dd-b0ac-eacabcf4414d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.834916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.835765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.836240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.836689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.847799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.862531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.862531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1063/2118] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:09.862531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:09.872551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d2a3-993b-42d4-b647-41398182c078 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:09.877904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ce51b-00d3-4eed-b986-29be51c8481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:09.878602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ce51b-00d3-4eed-b986-29be51c8481f admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.879148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ce51b-00d3-4eed-b986-29be51c8481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.879148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ce51b-00d3-4eed-b986-29be51c8481f admin admin] PATH_INFO: `/v3/projects/4fb37539d1c043ada3e70ab41526fb91/users/9bb7cd007d9041d7bffb96db1dbe04d4/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.892138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1056/2119] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:09 2022] PUT /identity/v3/projects/e9655193b10a47e6ba071e70d0d8b7aa/users/981d5bb01b35421ba8afe0df824ba5f6/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:09.902483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:09.902908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:09.903268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:09.904179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ce51b-00d3-4eed-b986-29be51c8481f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb37539d1c043ada3e70ab41526fb91, user_id=9bb7cd007d9041d7bffb96db1dbe04d4, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:09.907888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ce51b-00d3-4eed-b986-29be51c8481f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:09.941253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] MFA Rules not processed for user `981d5bb01b35421ba8afe0df824ba5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:09.942637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:10.027685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1057/2120] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:10.035738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:10.037248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:10.061618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1064/2121] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:09 2022] PUT /identity/v3/projects/4fb37539d1c043ada3e70ab41526fb91/users/9bb7cd007d9041d7bffb96db1dbe04d4/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:10.073738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:10.074207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:10.074582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:10.111821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b71e7-5eae-4411-8d41-4a4618bc0673 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:10.130709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] MFA Rules not processed for user `9bb7cd007d9041d7bffb96db1dbe04d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:10.132260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:10.181621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:10.182248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:10.182529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:10.182804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:10.241179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1065/2122] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:10.255386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:10.257071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:10.269968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:10.271577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:10.272927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836fe64-1e5e-43d8-89a1-193152fbaedd None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:10.277959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52b623-ec95-4720-8ba6-08183946f0bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:10.279040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52b623-ec95-4720-8ba6-08183946f0bd service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:10.279756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52b623-ec95-4720-8ba6-08183946f0bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:10.280709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52b623-ec95-4720-8ba6-08183946f0bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:10.286075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1058/2123] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:10.300296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52b623-ec95-4720-8ba6-08183946f0bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:10.300296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52b623-ec95-4720-8ba6-08183946f0bd service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:10.311109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1066/2124] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:10.691288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:11.186674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:11.473263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:11.484571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e1c9a-f7b3-4fd0-b8ca-050339bfcdae service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:11.495890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:11.496547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:11.496832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:11.497334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:11.499436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:11.573873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:11.576029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892388c-71ee-422d-95d2-6c9518edb599 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:11.602851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1059/2125] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:12.291170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:12.292090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:12.292376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:12.312299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] MFA Rules not processed for user `981d5bb01b35421ba8afe0df824ba5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:12.313505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:12.320501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1067/2126] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:12.725869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:13.230940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:13.333317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4cd9c-831b-40b0-b6fe-5994e6170d71 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:13.333603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4cd9c-831b-40b0-b6fe-5994e6170d71 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:13.333897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4cd9c-831b-40b0-b6fe-5994e6170d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:13.346063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:13.364682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec30417-ef80-4442-ac18-c1d4c7673d2a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:13.370487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:13.372042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:13.372172 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c8e4cd9c-831b-40b0-b6fe-5994e6170d71 None None] MFA Rules not processed for user `90f07b101240464387f3fbf92c925ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:13.373226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:13.373512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c8e4cd9c-831b-40b0-b6fe-5994e6170d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:13.373623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:13.380415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:13.449357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1060/2127] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:13.488363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:13.489876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:13.504576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1068/2128] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:14.295246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:14.369468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fa605-108e-44f8-b3a8-60cad550238c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:14.369784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fa605-108e-44f8-b3a8-60cad550238c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:14.370042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fa605-108e-44f8-b3a8-60cad550238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:14.389891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-670fa605-108e-44f8-b3a8-60cad550238c None None] MFA Rules not processed for user `6c886285823a45a0b921a180d5bf4f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:14.391443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-670fa605-108e-44f8-b3a8-60cad550238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:14.460084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1061/2129] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:15.113113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:15.195835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa8086-bfcb-4c27-89c3-f80dbbfcbca2 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:15.201382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:15.202181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:15.207413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:15.207928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] PATH_INFO: `/v3/users/9bb7cd007d9041d7bffb96db1dbe04d4` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:15.209345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb7cd007d9041d7bffb96db1dbe04d4)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:15.211337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:15.219829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:15.445877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb7cd007d9041d7bffb96db1dbe04d4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:15.564538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb7cd007d9041d7bffb96db1dbe04d4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:15.565143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb7cd007d9041d7bffb96db1dbe04d4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:15.567544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb7cd007d9041d7bffb96db1dbe04d4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:15.581631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb7cd007d9041d7bffb96db1dbe04d4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:15.620348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1069/2130] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:15 2022] DELETE /identity/v3/users/9bb7cd007d9041d7bffb96db1dbe04d4 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:15.625870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:15.626413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:15.626736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:15.650626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:15.651978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:15.751156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1062/2131] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:15.761269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:15.765991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:15.820760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30855a13-dc51-4222-9e47-669912b24955 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:15.824716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cdf6c-39a3-4e99-bfc5-1686f22d0c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:15.825325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cdf6c-39a3-4e99-bfc5-1686f22d0c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:15.825621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cdf6c-39a3-4e99-bfc5-1686f22d0c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:15.825888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cdf6c-39a3-4e99-bfc5-1686f22d0c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:15.835380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cdf6c-39a3-4e99-bfc5-1686f22d0c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:15.836807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cdf6c-39a3-4e99-bfc5-1686f22d0c90 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:15.846794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1070/2132] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:16.252018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:16.281917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:16.284413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:16.296849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767b55c-cad4-4f05-bae9-7d5d4522d3c8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:16.301675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:16.302532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:16.302694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:16.303025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] PATH_INFO: `/v3/projects/4fb37539d1c043ada3e70ab41526fb91` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:16.304385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb37539d1c043ada3e70ab41526fb91)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:16.306534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:16.450046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb37539d1c043ada3e70ab41526fb91', 'request_id': 'req-5187330d-a6d0-4cb5-b79e-2fe38d78c287'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:16.456355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb37539d1c043ada3e70ab41526fb91', 'request_id': 'req-5187330d-a6d0-4cb5-b79e-2fe38d78c287'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:16.525107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb37539d1c043ada3e70ab41526fb91', 'request_id': 'req-5187330d-a6d0-4cb5-b79e-2fe38d78c287'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:16.536949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] The token cache is being invalidate because project 4fb37539d1c043ada3e70ab41526fb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:16.537249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:16.537896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5187330d-a6d0-4cb5-b79e-2fe38d78c287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:16.539480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1063/2133] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:16 2022] DELETE /identity/v3/projects/4fb37539d1c043ada3e70ab41526fb91 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:16.544580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:16.544866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:16.545139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:16.567204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:16.568681 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:16.657635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1071/2134] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:16.671903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:16.672814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:16.673093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:16.706455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:16.708105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:16.727298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1064/2135] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:16.730436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:16.747441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c540b03-9f0b-4b00-90c5-8aefb03defb9 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:16.759582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:16.760423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:16.760718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:16.760997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:16.761683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:16.761996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:16.763903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:16.797450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1072/2136] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:16.802196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:16.817073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005e07-3a70-41d3-9f68-f99c741c8fbd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:16.822655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:16.825669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:16.825669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:16.825669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:16.825669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:16.826515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:17.039105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1065/2137] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:16 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:17.043264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:17.053957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1697b6-dd37-4ccb-8c6f-ea1ff1ee0fdc admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:17.160501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:17.160590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:17.160590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.160590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.160811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.161446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:17.163694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:17.284560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1073/2138] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:17 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:17.289088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:17.301788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef818b60-7c13-45cc-9e0a-3da964d694aa admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:17.308111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:17.309082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.309783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.310967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.312007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:17.314140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:17.326411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1066/2139] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:17:17.331134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:17.341538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44ba49-c36a-480d-9e8c-c69fd15ab1fd admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:17.352386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49f783-d312-411d-af70-00f21c10464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:17.353066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f783-d312-411d-af70-00f21c10464d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.353350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f783-d312-411d-af70-00f21c10464d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.353626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f783-d312-411d-af70-00f21c10464d admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.354338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49f783-d312-411d-af70-00f21c10464d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:17.356171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49f783-d312-411d-af70-00f21c10464d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:17.367943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1074/2140] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:17.372143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:17.382935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e433b94-a869-4a74-8aea-47ad6027569a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:17.387717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:17.388388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.388671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.388949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] PATH_INFO: `/v3/projects/d190b110f5cf45d7947e46832f6f76a5/users/5eab97732ccf4eb0b9304f04ece2d0e6/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.413217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d190b110f5cf45d7947e46832f6f76a5, user_id=5eab97732ccf4eb0b9304f04ece2d0e6, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:17.416944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:17.522232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1067/2141] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:17 2022] PUT /identity/v3/projects/d190b110f5cf45d7947e46832f6f76a5/users/5eab97732ccf4eb0b9304f04ece2d0e6/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:17.534308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.534899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.535208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.578349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] MFA Rules not processed for user `5eab97732ccf4eb0b9304f04ece2d0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:17.578349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:17.640859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1075/2142] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:17.645571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:17.712554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f9569-a7c3-46ad-bc93-1100deb297f4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:17.717346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:17.718006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.718800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.719074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.719761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:17.721665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:17.818985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1068/2143] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:17 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:17.823664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:17.845236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ac07-1d8e-412b-9bec-ba4b3078b650 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:17.964709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:17.965346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:17.965633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:17.965905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:17.966612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:17.969141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:18.083859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1076/2144] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:17 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:18.087849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:18.099137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1198-6a72-40d7-86f8-3cd6a7c1a0a3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:18.104068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:18.104754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.105040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.105314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.105956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:18.107581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:18.119055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1069/2145] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:18.123199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:18.134707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddcc81-77e7-46b9-b22e-5de0584672d6 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:18.141990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1810b-4391-4936-9c42-282e28c055f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:18.142730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1810b-4391-4936-9c42-282e28c055f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.143081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1810b-4391-4936-9c42-282e28c055f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.143407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1810b-4391-4936-9c42-282e28c055f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.144194 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1810b-4391-4936-9c42-282e28c055f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:18.145769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1810b-4391-4936-9c42-282e28c055f7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:18.157050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1077/2146] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:18.161364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:18.171859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270a1a0-0a26-4abe-bd39-aa843bdb9598 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:18.185942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:18.186812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.187134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.187577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] PATH_INFO: `/v3/projects/25c5707224bb4b84a6566038b33ad8b2/users/38c9e1cf89064ba2a35196dbbb98b662/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.209177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c5707224bb4b84a6566038b33ad8b2, user_id=38c9e1cf89064ba2a35196dbbb98b662, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:18.212360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:18.416435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:18.422962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1070/2147] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:18 2022] PUT /identity/v3/projects/25c5707224bb4b84a6566038b33ad8b2/users/38c9e1cf89064ba2a35196dbbb98b662/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:18.435603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.435894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.436171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.489830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:18.499345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] MFA Rules not processed for user `38c9e1cf89064ba2a35196dbbb98b662`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:18.499345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:18.602202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1078/2148] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:18.607690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:18.708764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:18.710704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:18.722250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980f8c7-799c-4d0a-852c-4aacf0593784 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:18.748573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:18.750680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.752832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.753762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.754548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:18.756675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:18.765396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dd079-723f-43d6-9273-11420e62fe33 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:18.784866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:18.785536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.785990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.786563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] PATH_INFO: `/v3/users/cabadbcddd854aa3812f6c0e8989064f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.788178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cabadbcddd854aa3812f6c0e8989064f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:18.790305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:18.873654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1071/2149] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:18 2022] POST /identity/v3/projects => generated 383 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:18.878370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:18.978656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1d7c2-2f50-4369-9439-f10c3304c17f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:18.985441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:18.986204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:18.986716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:18.988277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:18.989277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:18.991503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:19.133019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1072/2150] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:18 2022] POST /identity/v3/users => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:19.137450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:19.154216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb5bef-3ef4-405d-ae59-ed147ba86d75 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:19.166997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:19.169117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:19.169117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:19.169117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:19.170156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:19.172888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:19.187863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1073/2151] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:19.188633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cabadbcddd854aa3812f6c0e8989064f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:19.192402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:19.202454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac35f60-37c9-4c09-85ab-e07d9f9c9f5b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:19.208208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:19.208886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:19.209165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:19.209446 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] PATH_INFO: `/v3/projects/24554b13b7aa45e084ebd5a09f73d06c/users/11207c056744428caa468bfa7b078a5b/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:19.226630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=24554b13b7aa45e084ebd5a09f73d06c, user_id=11207c056744428caa468bfa7b078a5b, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:19.229371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:19.306988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cabadbcddd854aa3812f6c0e8989064f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:19.308106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cabadbcddd854aa3812f6c0e8989064f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:19.309347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cabadbcddd854aa3812f6c0e8989064f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:19.329347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34aaecc8-51ca-48ef-8508-0f81a9dcf6b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cabadbcddd854aa3812f6c0e8989064f'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:19.333807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1074/2152] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:19 2022] PUT /identity/v3/projects/24554b13b7aa45e084ebd5a09f73d06c/users/11207c056744428caa468bfa7b078a5b/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:19.338280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:19.341655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:19.389716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1079/2153] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:18 2022] DELETE /identity/v3/users/cabadbcddd854aa3812f6c0e8989064f => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:17:19.398044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:19.398511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:19.398980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:19.459716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:19.465949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:19.535615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:19.551637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae72590-93e0-4c8d-a08f-3758765c55bb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:19.653612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1080/2154] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:19.675441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:19.676287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:19.676577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:19.676914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:19.677598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:19.679302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:19.690780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:19.695291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:19.695497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1075/2155] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:19.696729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:19.710669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cb16e-e58f-4a14-aef7-8586cbe2988a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:19.720584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce299-22f1-442b-b29d-06fa05849d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:19.721910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce299-22f1-442b-b29d-06fa05849d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:19.723444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce299-22f1-442b-b29d-06fa05849d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:19.723444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce299-22f1-442b-b29d-06fa05849d44 admin admin] PATH_INFO: `/v3/projects/24554b13b7aa45e084ebd5a09f73d06c/users/11207c056744428caa468bfa7b078a5b/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:19.727052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce299-22f1-442b-b29d-06fa05849d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24554b13b7aa45e084ebd5a09f73d06c, user_id=11207c056744428caa468bfa7b078a5b, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:19.733566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce299-22f1-442b-b29d-06fa05849d44 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:19.788911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1076/2156] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:19 2022] PUT /identity/v3/projects/24554b13b7aa45e084ebd5a09f73d06c/users/11207c056744428caa468bfa7b078a5b/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:19.803550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:19.803827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:19.806233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:19.868774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcf055-a144-4676-b8f8-cb0c72f7fc18 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:19.885520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] MFA Rules not processed for user `11207c056744428caa468bfa7b078a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:19.887106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:20.003677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1077/2157] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:20.016660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:20.019254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:20.020861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:20.021600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:20.021985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:20.022377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:20.102246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:20.103779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:20.119113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1081/2158] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:20.132026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7951572-6e29-43fc-aa49-100d16ac4c53 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:20.136293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:20.136922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:20.137205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:20.137479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:20.139275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:20.272237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:20.275259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:20.298670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1078/2159] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:20.963834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:20.965331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:20.982625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d26ad3-5643-441d-8b9e-59367d3a93b6 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:20.991749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:20.992735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:20.993015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:20.993473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] PATH_INFO: `/v3/projects/17ee24894ef8439b9a81f8336ce8a0ff` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:20.995066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ee24894ef8439b9a81f8336ce8a0ff)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:20.997484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:21.155729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ee24894ef8439b9a81f8336ce8a0ff', 'request_id': 'req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.172942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ee24894ef8439b9a81f8336ce8a0ff', 'request_id': 'req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.279988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ee24894ef8439b9a81f8336ce8a0ff', 'request_id': 'req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.329370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] The token cache is being invalidate because project 17ee24894ef8439b9a81f8336ce8a0ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:21.330127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.332493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.338871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1082/2160] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:20 2022] DELETE /identity/v3/projects/17ee24894ef8439b9a81f8336ce8a0ff => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:21.339599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:21.353834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8e2e1-da34-406e-a979-993182047e67 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:21.429128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:21.429784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:21.430065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:21.430342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] PATH_INFO: `/v3/users/47f799a34fd84eb8b36cfe196f174520` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:21.431522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f799a34fd84eb8b36cfe196f174520)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:21.433547 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:21.713641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:21.757783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f799a34fd84eb8b36cfe196f174520'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.833095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f799a34fd84eb8b36cfe196f174520'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.833716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f799a34fd84eb8b36cfe196f174520'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.834314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f799a34fd84eb8b36cfe196f174520'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.841702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f799a34fd84eb8b36cfe196f174520'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:21.858627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1079/2161] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:21 2022] DELETE /identity/v3/users/47f799a34fd84eb8b36cfe196f174520 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:22.439551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:22.541247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:22.542934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:22.557058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5f41c-5ee5-42a8-b50b-ab175cdde1ee admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:22.656635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:22.657688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:22.658085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:22.658510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] PATH_INFO: `/v3/projects/1d14d87770a34979939c7978384d0f67` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:22.660841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d14d87770a34979939c7978384d0f67)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:22.662962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:22.711310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:22.713055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:22.779780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ba48-f9b1-45f4-86f2-d92ea81f840c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:22.779780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:22.781850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:22.782227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:22.782732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:22.784961 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:22.819460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d14d87770a34979939c7978384d0f67', 'request_id': 'req-21d69dcb-5e9b-49fe-a56e-c60d2530f049'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:22.830373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d14d87770a34979939c7978384d0f67', 'request_id': 'req-21d69dcb-5e9b-49fe-a56e-c60d2530f049'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:22.851184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:22.852714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:22.862742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1080/2162] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:22.891263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d14d87770a34979939c7978384d0f67', 'request_id': 'req-21d69dcb-5e9b-49fe-a56e-c60d2530f049'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:22.906602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] The token cache is being invalidate because project 1d14d87770a34979939c7978384d0f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:22.907066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:22.907978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:22.911200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1083/2163] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:22 2022] DELETE /identity/v3/projects/1d14d87770a34979939c7978384d0f67 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:22.913983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:22.915779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:22.930704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cc557-167d-43eb-8841-58e056bdb919 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:22.991401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:22.992063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:22.992343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:22.992614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] PATH_INFO: `/v3/users/82bf7775c3e14531afeb30740de507c6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:22.996084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bf7775c3e14531afeb30740de507c6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:22.998034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:23.207827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bf7775c3e14531afeb30740de507c6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:23.291641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bf7775c3e14531afeb30740de507c6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:23.292578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bf7775c3e14531afeb30740de507c6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:23.293491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bf7775c3e14531afeb30740de507c6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:23.309246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ec50e32-3074-46df-b655-571a7029583d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bf7775c3e14531afeb30740de507c6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:23.332063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1081/2164] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:22 2022] DELETE /identity/v3/users/82bf7775c3e14531afeb30740de507c6 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:23.674320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:23.771620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:23.813025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:23.815086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:23.832939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d69dcb-5e9b-49fe-a56e-c60d2530f049 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:23.896999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:23.897564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:23.897924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:23.898245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] PATH_INFO: `/v3/projects/ee404cbdccb04607993a4d310a36378b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:23.899414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee404cbdccb04607993a4d310a36378b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:23.901374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.044629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee404cbdccb04607993a4d310a36378b', 'request_id': 'req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:24.050151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee404cbdccb04607993a4d310a36378b', 'request_id': 'req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:24.168781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee404cbdccb04607993a4d310a36378b', 'request_id': 'req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:24.180966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] The token cache is being invalidate because project ee404cbdccb04607993a4d310a36378b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:24.182211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:24.182211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2bcafcef-5978-4c5f-a5db-7ed2905f41d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:24.184278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1084/2165] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:23 2022] DELETE /identity/v3/projects/ee404cbdccb04607993a4d310a36378b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:24.189954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.190283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.190591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.212309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:24.213622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:24.283342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1082/2166] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:24.286311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.286800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.287084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.309192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:24.310129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:24.323055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1085/2167] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:24.328263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.340747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666762-0f44-480f-a6fe-d6562000bd62 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.348860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.349531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.349906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.350251 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.352136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:24.352518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.354287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.371212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1083/2168] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:24.373271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.392007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29312cee-4afd-4ec3-b44e-57e1d759e367 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.400507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.401188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.401467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.401739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.402395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.404827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.482625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.485340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:24.501082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1086/2169] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:24 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:24.512207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.602293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9490-3b0c-4a8a-9f02-a6878d97f551 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.607204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5158c-804d-4ebc-8126-dbd7486f2a0b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.613213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.613952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.614338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.614715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.615512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.617585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.658761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.660467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.660467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.660467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.661960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:24.717689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1087/2170] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:24 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:24.719661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.729726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.730767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45078d6-fc60-408d-9e9f-493a9f81f052 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.731264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.736368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.737183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.741032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.741032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.741032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.741032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.743218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1084/2171] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:17:24.751026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1088/2172] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:24.755065 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.766076 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f12c-9d1c-4096-8f95-8cbad2920219 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.770810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1884-028d-4e92-a716-555d4d32be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.771439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1884-028d-4e92-a716-555d4d32be1c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.771710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1884-028d-4e92-a716-555d4d32be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.772226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1884-028d-4e92-a716-555d4d32be1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.772587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1884-028d-4e92-a716-555d4d32be1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.774188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1884-028d-4e92-a716-555d4d32be1c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.785008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1085/2173] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:17:24.789353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:24.801888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e5603-aa97-4d7e-af9a-345746db00d3 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:24.806517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:24.807205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.807465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.807739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] PATH_INFO: `/v3/projects/1dd46de623ca4fafafe6f052c451982c/users/d542a0d1d16f466288270a8a8ee4c2b4/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.823053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd46de623ca4fafafe6f052c451982c, user_id=d542a0d1d16f466288270a8a8ee4c2b4, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:24.826025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:24.902174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1089/2174] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:24 2022] PUT /identity/v3/projects/1dd46de623ca4fafafe6f052c451982c/users/d542a0d1d16f466288270a8a8ee4c2b4/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:24.913502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:24.913790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:24.914067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:24.941984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] MFA Rules not processed for user `d542a0d1d16f466288270a8a8ee4c2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:24.943240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:24.994892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:25.010058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1086/2175] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:25.023046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:25.027311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:25.037149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:25.105545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc28651-3e64-4531-aabc-8307addc0984 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:25.110148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd87e8-dcae-43c9-a753-f4414e41bce0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:25.110835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd87e8-dcae-43c9-a753-f4414e41bce0 service glance] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:25.111040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd87e8-dcae-43c9-a753-f4414e41bce0 service glance] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:25.111346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd87e8-dcae-43c9-a753-f4414e41bce0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:25.120027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd87e8-dcae-43c9-a753-f4414e41bce0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:25.121468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd87e8-dcae-43c9-a753-f4414e41bce0 service glance] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:25.130550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1090/2176] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:27.308373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:27.433902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:28.494494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:29.116654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:29.754255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:29.755738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:29.823533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f039a-e58c-47fc-914e-d4337e07092b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:29.828160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:29.829844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:29.830128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:29.830409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] PATH_INFO: `/v3/users/5eab97732ccf4eb0b9304f04ece2d0e6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:29.831634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eab97732ccf4eb0b9304f04ece2d0e6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:29.836478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:30.091386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eab97732ccf4eb0b9304f04ece2d0e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:30.158735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eab97732ccf4eb0b9304f04ece2d0e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:30.159386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eab97732ccf4eb0b9304f04ece2d0e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:30.160002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eab97732ccf4eb0b9304f04ece2d0e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:30.167828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eab97732ccf4eb0b9304f04ece2d0e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:30.192952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1087/2177] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:29 2022] DELETE /identity/v3/users/5eab97732ccf4eb0b9304f04ece2d0e6 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:17:30.197421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:30.197940 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:30.198030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:30.218773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:30.220686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:30.302380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:30.306352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1091/2178] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:30.316167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:30.318191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:30.405931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e60fe4-a6ab-4219-b4b2-810a898f0cba admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:30.410751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:30.411888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:30.412376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:30.412853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:30.425600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:30.432766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:30.437752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1088/2179] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:30.592287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:31.062202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:31.064786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:31.088961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64d41-4fff-44cd-8461-8549b68a1628 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:31.104756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:31.104756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:31.104756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:31.104756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] PATH_INFO: `/v3/projects/d190b110f5cf45d7947e46832f6f76a5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:31.105992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d190b110f5cf45d7947e46832f6f76a5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:31.109507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:31.271139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd190b110f5cf45d7947e46832f6f76a5', 'request_id': 'req-c8011f47-1216-4796-991d-37daaa635922'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.292247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd190b110f5cf45d7947e46832f6f76a5', 'request_id': 'req-c8011f47-1216-4796-991d-37daaa635922'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.393356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd190b110f5cf45d7947e46832f6f76a5', 'request_id': 'req-c8011f47-1216-4796-991d-37daaa635922'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.411767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] The token cache is being invalidate because project d190b110f5cf45d7947e46832f6f76a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:31.412306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.413130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.419538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:31.420060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1092/2180] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:31 2022] DELETE /identity/v3/projects/d190b110f5cf45d7947e46832f6f76a5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:31.430004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936b0f3-68c7-4140-b302-516b2d45655f service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:31.491424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:31.492097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:31.492381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:31.492658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] PATH_INFO: `/v3/users/38c9e1cf89064ba2a35196dbbb98b662` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:31.493834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c9e1cf89064ba2a35196dbbb98b662)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:31.495807 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:31.699437 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c9e1cf89064ba2a35196dbbb98b662'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.703399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:31.810825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c9e1cf89064ba2a35196dbbb98b662'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.811568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c9e1cf89064ba2a35196dbbb98b662'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.812225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c9e1cf89064ba2a35196dbbb98b662'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.819785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c9e1cf89064ba2a35196dbbb98b662'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:31.852830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1089/2181] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:31 2022] DELETE /identity/v3/users/38c9e1cf89064ba2a35196dbbb98b662 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:32.485112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:32.487037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:32.505536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8011f47-1216-4796-991d-37daaa635922 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:32.600697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:32.600697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:32.600697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:32.600697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] PATH_INFO: `/v3/projects/25c5707224bb4b84a6566038b33ad8b2` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:32.601195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c5707224bb4b84a6566038b33ad8b2)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:32.602013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:32.709620 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:32.718030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c5707224bb4b84a6566038b33ad8b2', 'request_id': 'req-c741a805-b1ad-448a-b7ae-c1436492f253'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:32.726503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c5707224bb4b84a6566038b33ad8b2', 'request_id': 'req-c741a805-b1ad-448a-b7ae-c1436492f253'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:32.824999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c5707224bb4b84a6566038b33ad8b2', 'request_id': 'req-c741a805-b1ad-448a-b7ae-c1436492f253'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:32.836823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] The token cache is being invalidate because project 25c5707224bb4b84a6566038b33ad8b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:32.837062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:32.837700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:32.839491 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1093/2182] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:32 2022] DELETE /identity/v3/projects/25c5707224bb4b84a6566038b33ad8b2 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:32.843298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:32.844857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:32.857484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ebd8f-35eb-401b-8da8-f36d753b5a6f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:32.914818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:32.915519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:32.915805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:32.916079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] PATH_INFO: `/v3/users/11207c056744428caa468bfa7b078a5b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:32.917380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11207c056744428caa468bfa7b078a5b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:32.919363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:33.085805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11207c056744428caa468bfa7b078a5b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.151243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11207c056744428caa468bfa7b078a5b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.151821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11207c056744428caa468bfa7b078a5b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.152460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11207c056744428caa468bfa7b078a5b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.162972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fd2d490-bfaa-4ccc-b0d1-9d215f3d5b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11207c056744428caa468bfa7b078a5b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.182193 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1090/2183] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:32 2022] DELETE /identity/v3/users/11207c056744428caa468bfa7b078a5b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:33.633624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:33.635263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:33.659300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741a805-b1ad-448a-b7ae-c1436492f253 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:33.720227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:33.720879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:33.721158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:33.721431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] PATH_INFO: `/v3/projects/24554b13b7aa45e084ebd5a09f73d06c` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:33.722616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24554b13b7aa45e084ebd5a09f73d06c)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:33.724619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:33.847836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24554b13b7aa45e084ebd5a09f73d06c', 'request_id': 'req-32073f92-cb88-4dff-b5ae-9c553d444b71'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.855242 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24554b13b7aa45e084ebd5a09f73d06c', 'request_id': 'req-32073f92-cb88-4dff-b5ae-9c553d444b71'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.930434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24554b13b7aa45e084ebd5a09f73d06c', 'request_id': 'req-32073f92-cb88-4dff-b5ae-9c553d444b71'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.942375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] The token cache is being invalidate because project 24554b13b7aa45e084ebd5a09f73d06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:33.942701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.943434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-32073f92-cb88-4dff-b5ae-9c553d444b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:33.944930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1094/2184] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:33 2022] DELETE /identity/v3/projects/24554b13b7aa45e084ebd5a09f73d06c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:33.950205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:33.950752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:33.950875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:33.972833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:33.974063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:34.019579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:34.034978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1091/2185] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:34.044145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.044435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.044754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.065808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:34.067561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:34.075929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1095/2186] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:34.080737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.090428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f20769-e1e4-4227-8e59-cf595020332d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.095152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.095788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.096069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.096342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.096991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:34.097297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.098924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.110868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1092/2187] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:34.115597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.125972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb32e5-0beb-4349-a744-8afa9d762ec3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.130769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.131401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.135288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.135288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.135288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.135288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.210183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1096/2188] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:34 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:34.211263 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.221472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154100a-639f-41f6-b3d1-7aa88a565eff admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.285101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.285826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.286175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.286517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.287468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.290014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.406809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1093/2189] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:34 2022] POST /identity/v3/users => generated 286 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:34.408452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.419026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ea05c-a4b7-4fcb-9d56-b1f928b702c6 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.424392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.424996 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.425279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.425554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.426254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.427894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.438071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1097/2190] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:34.442232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.452463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05b9dc-d6f4-4129-a36c-cb07a02b29a9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.457874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31120c-b327-4e8e-964c-bdcda7361914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.458812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31120c-b327-4e8e-964c-bdcda7361914 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.459273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31120c-b327-4e8e-964c-bdcda7361914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.459763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31120c-b327-4e8e-964c-bdcda7361914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.460618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31120c-b327-4e8e-964c-bdcda7361914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.462629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31120c-b327-4e8e-964c-bdcda7361914 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.475798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1094/2191] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:34.480038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.494135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd69b-3ce5-4ef5-a2e9-2f5e85197f35 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.498909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.499602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.499886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.500305 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] PATH_INFO: `/v3/projects/eaf22fc86ccc42fb94ed40c592330133/users/fd899fb668084a39aff57b2f38498d16/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.513117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:34.519833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf22fc86ccc42fb94ed40c592330133, user_id=fd899fb668084a39aff57b2f38498d16, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.522839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.608711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1098/2192] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:34 2022] PUT /identity/v3/projects/eaf22fc86ccc42fb94ed40c592330133/users/fd899fb668084a39aff57b2f38498d16/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:34.619557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d200b-6a76-41c8-8871-0530a8152ba9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.619913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d200b-6a76-41c8-8871-0530a8152ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.620122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d200b-6a76-41c8-8871-0530a8152ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.650656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3d3d200b-6a76-41c8-8871-0530a8152ba9 None None] MFA Rules not processed for user `fd899fb668084a39aff57b2f38498d16`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:34.652008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3d3d200b-6a76-41c8-8871-0530a8152ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:34.727560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1095/2193] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:34.739478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:34.741307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:34.803964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57716d94-4749-4256-989e-2d5c1c6d88dc admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:34.807864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:34.808302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:34.808585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:34.808858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:34.820939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:34.822528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:34.832611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1099/2194] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:41.020775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555931b0-b37b-4bfa-83a9-cc7fdf1de20e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:41.021314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555931b0-b37b-4bfa-83a9-cc7fdf1de20e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:41.021538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555931b0-b37b-4bfa-83a9-cc7fdf1de20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:41.047054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-555931b0-b37b-4bfa-83a9-cc7fdf1de20e None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:41.048953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-555931b0-b37b-4bfa-83a9-cc7fdf1de20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:41.100077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:41.114953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1096/2195] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:41.159630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:41.161126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:41.234677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c737a-e24a-47ea-a128-a9f08313daec service glance] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:41.240070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:41.240674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:41.240950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:41.241219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:41.249721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:41.251336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:41.259392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1100/2196] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:17:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:17:42.185542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650aef-c172-42a4-9b28-117bc2c7cb99 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:42.186015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650aef-c172-42a4-9b28-117bc2c7cb99 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:42.186106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650aef-c172-42a4-9b28-117bc2c7cb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:42.209840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3f650aef-c172-42a4-9b28-117bc2c7cb99 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:42.211068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3f650aef-c172-42a4-9b28-117bc2c7cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:42.218338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1097/2197] 10.222.0.62 () {58 vars in 967 bytes} [Tue Sep 13 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:42.236740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:42.246385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbdbf0-f73a-4e0d-85ae-ff8eab8fecb5 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:42.254186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:42.254990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:42.255506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:42.256153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:42.267028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:42.269384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:42.279666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1101/2198] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:17:42.907831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa1ba-8d38-4222-b081-f45c11101a31 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:42.908117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa1ba-8d38-4222-b081-f45c11101a31 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:42.911644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa1ba-8d38-4222-b081-f45c11101a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:42.927444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-db0aa1ba-8d38-4222-b081-f45c11101a31 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:42.929303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-db0aa1ba-8d38-4222-b081-f45c11101a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:42.936514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1098/2199] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:42.949754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:42.961567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175dd7-7301-4c59-a4c1-fe99c8126e91 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:42.966573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:42.967192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:42.967542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:42.967812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:42.976236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:42.977833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:42.986123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1102/2200] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:17:43.577871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:43.578330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:43.578416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:43.600139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:43.601684 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:43.615983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1099/2201] 10.222.0.62 () {58 vars in 967 bytes} [Tue Sep 13 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:43.623148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:43.634546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984283-162d-4e4d-8f7e-30105768a867 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:43.639688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280259c-3a7f-467e-b563-f10c340381c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:43.640292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280259c-3a7f-467e-b563-f10c340381c2 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:43.640609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280259c-3a7f-467e-b563-f10c340381c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:43.640849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280259c-3a7f-467e-b563-f10c340381c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:43.649727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280259c-3a7f-467e-b563-f10c340381c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:43.651348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280259c-3a7f-467e-b563-f10c340381c2 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:43.660536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1103/2202] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:17:46.561707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:46.563200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:46.644323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6c838-bd70-419f-9642-1b0b83fa5b39 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:46.649411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:46.650052 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:46.650332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:46.650603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] PATH_INFO: `/v3/users/981d5bb01b35421ba8afe0df824ba5f6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:46.651790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=981d5bb01b35421ba8afe0df824ba5f6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:46.653699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:46.853087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '981d5bb01b35421ba8afe0df824ba5f6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:46.946909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '981d5bb01b35421ba8afe0df824ba5f6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:46.947568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '981d5bb01b35421ba8afe0df824ba5f6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:46.948192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '981d5bb01b35421ba8afe0df824ba5f6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:46.955855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '981d5bb01b35421ba8afe0df824ba5f6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:46.974245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1100/2203] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:17:46 2022] DELETE /identity/v3/users/981d5bb01b35421ba8afe0df824ba5f6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:46.978789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:46.979362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:46.979461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:46.999145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:47.000537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:47.054076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:47.073566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1104/2204] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:47.089198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:47.090713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:47.288196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:47.289421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e56416-32a0-48f9-a83d-ffd7af288440 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:47.293201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793aaa7f-cd5b-48ca-be34-eff67b19f425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:47.294106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793aaa7f-cd5b-48ca-be34-eff67b19f425 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:47.294523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793aaa7f-cd5b-48ca-be34-eff67b19f425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:47.294869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793aaa7f-cd5b-48ca-be34-eff67b19f425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:47.309748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793aaa7f-cd5b-48ca-be34-eff67b19f425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:47.309748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793aaa7f-cd5b-48ca-be34-eff67b19f425 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:47.317153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1101/2205] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:47.869304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:47.871295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:47.884379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be656e4c-ada4-4702-b7e0-5273a8b39c5c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:47.889492 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:47.890144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:47.890425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:47.890698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] PATH_INFO: `/v3/projects/e9655193b10a47e6ba071e70d0d8b7aa` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:47.891889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9655193b10a47e6ba071e70d0d8b7aa)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:47.893819 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:47.936970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed4f89-c1aa-49fb-830a-fa4e296c48c5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:47.937453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed4f89-c1aa-49fb-830a-fa4e296c48c5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:47.937550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed4f89-c1aa-49fb-830a-fa4e296c48c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:47.956832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f8ed4f89-c1aa-49fb-830a-fa4e296c48c5 None None] MFA Rules not processed for user `4067d48268f84854af4d672ab3f9dc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:47.958175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f8ed4f89-c1aa-49fb-830a-fa4e296c48c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:48.025402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9655193b10a47e6ba071e70d0d8b7aa', 'request_id': 'req-5d86d162-2f3c-483b-bc5e-22681b939ce9'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:48.033005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9655193b10a47e6ba071e70d0d8b7aa', 'request_id': 'req-5d86d162-2f3c-483b-bc5e-22681b939ce9'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:48.035295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1102/2206] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:48.150435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9655193b10a47e6ba071e70d0d8b7aa', 'request_id': 'req-5d86d162-2f3c-483b-bc5e-22681b939ce9'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:48.163238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] The token cache is being invalidate because project e9655193b10a47e6ba071e70d0d8b7aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:17:48.163613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:48.164328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-5d86d162-2f3c-483b-bc5e-22681b939ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:17:48.166603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1105/2207] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:17:47 2022] DELETE /identity/v3/projects/e9655193b10a47e6ba071e70d0d8b7aa => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:48.171605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.173441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.173441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.199543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:48.199543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:48.277361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1103/2208] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:48.286851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.287139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.287450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.309624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:48.311416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:48.321757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1106/2209] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:48.327430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:48.338069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0a437-9284-4db5-8ef7-a46519aa3288 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:48.353246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:48.353246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.353246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.353246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.353614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:17:48.353614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:48.353614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:48.363334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1104/2210] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:17:48.371630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:48.385449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ca1c6-6a8e-4281-98f7-d45458cf5c28 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:48.394570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:48.395511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.396100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.397224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.397989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:48.400647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:48.516932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1107/2211] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:48 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:17:48.518151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:48.534247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697545f-cd73-486a-9990-e64fe33b0eb8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:48.615378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:48.618654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.618654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.618654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.618654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:48.619291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:48.708689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1105/2212] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:48 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:48.710035 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:48.721030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bceb82-9853-4899-8762-467c9d44fc96 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:48.725672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:48.726281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.726563 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.726905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.727512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:48.729113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:48.745752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1108/2213] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:48.753569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:48.765630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ef11-ed85-44a5-9b31-52a11c64e807 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:48.773314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eef322-1e25-430b-b3f7-e89b718463e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:48.773995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eef322-1e25-430b-b3f7-e89b718463e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.774348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eef322-1e25-430b-b3f7-e89b718463e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.774619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eef322-1e25-430b-b3f7-e89b718463e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.775467 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eef322-1e25-430b-b3f7-e89b718463e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:48.776876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eef322-1e25-430b-b3f7-e89b718463e3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:48.795770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1106/2214] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:48.807263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:48.818870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3e7c-3171-41f6-8b89-c52ddd6358a9 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:48.824268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:48.824936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.825234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.825582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] PATH_INFO: `/v3/projects/7cc199dd87e24511a8d267603b571456/users/7ddb2572cac446aa92a0bcf170d9e8d6/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.847451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc199dd87e24511a8d267603b571456, user_id=7ddb2572cac446aa92a0bcf170d9e8d6, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:48.848494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:48.929756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1109/2215] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:48 2022] PUT /identity/v3/projects/7cc199dd87e24511a8d267603b571456/users/7ddb2572cac446aa92a0bcf170d9e8d6/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:48.935200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e290f-d09d-4373-80db-39b5bf17926b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:48.935200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e290f-d09d-4373-80db-39b5bf17926b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:48.938548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e290f-d09d-4373-80db-39b5bf17926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:48.952608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:48.958742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-be5e290f-d09d-4373-80db-39b5bf17926b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:48.960129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-be5e290f-d09d-4373-80db-39b5bf17926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:49.022009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1107/2216] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:49.051219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:49.053175 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:49.215151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b016b-909c-4862-93e4-270a84f829bc admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:49.219480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:49.220414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:49.220847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:49.221392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:49.237100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:49.238697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:49.278200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1110/2217] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:17:55.159897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.160582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.160862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.200381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] MFA Rules not processed for user `7ddb2572cac446aa92a0bcf170d9e8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:55.201653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:55.272912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1108/2218] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:17:55.277892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:55.290398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbdb6a-ec13-439f-bb63-ef9b218eddcf service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:55.295945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:55.302847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.302847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.302847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.302847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:55.302847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:55.306235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:55.401110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1111/2219] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:17:55 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:55.403643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:55.417434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568ba2-a481-4f90-9d32-4a7776ef44a8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:55.477857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:55.478644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.478925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.479234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.480101 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:55.482045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:55.605042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1109/2220] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:17:55 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:17:55.609076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:55.619872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923b058-9b27-4f4d-ae59-0433b5611652 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:55.624913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:55.625614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.625899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.626203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.626856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:55.628503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:55.639430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1112/2221] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:17:55.644177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:55.663271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7352-8aee-4109-ba94-6c240c5c8bad admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:55.663271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:55.663271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.663271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.663711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] PATH_INFO: `/v3/projects/e6e686683db641109e4b1ed65cbbd3a1/users/6dcd84cc95cc4a899133c320ed7d4209/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.676877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e686683db641109e4b1ed65cbbd3a1, user_id=6dcd84cc95cc4a899133c320ed7d4209, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:55.679614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:55.770646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:17:55.776247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1110/2222] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:55 2022] PUT /identity/v3/projects/e6e686683db641109e4b1ed65cbbd3a1/users/6dcd84cc95cc4a899133c320ed7d4209/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:55.780466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:55.790476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d509c3-f2f1-491c-b563-32966cee4189 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:55.871725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553bafa-9312-43ae-9dec-1997843e1fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:55.872350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553bafa-9312-43ae-9dec-1997843e1fce admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.872618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553bafa-9312-43ae-9dec-1997843e1fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.872951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553bafa-9312-43ae-9dec-1997843e1fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.873625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553bafa-9312-43ae-9dec-1997843e1fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:55.875793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553bafa-9312-43ae-9dec-1997843e1fce admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:55.885475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1113/2223] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:17:55.889761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:55.900582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa613f1-ca30-4243-820f-afc704bc68fb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:17:55.905472 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797c9ae-d95f-4caf-a351-e52875748537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:17:55.906125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797c9ae-d95f-4caf-a351-e52875748537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:55.911934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797c9ae-d95f-4caf-a351-e52875748537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:55.911934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797c9ae-d95f-4caf-a351-e52875748537 admin admin] PATH_INFO: `/v3/projects/e6e686683db641109e4b1ed65cbbd3a1/users/6dcd84cc95cc4a899133c320ed7d4209/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:55.911934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797c9ae-d95f-4caf-a351-e52875748537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e686683db641109e4b1ed65cbbd3a1, user_id=6dcd84cc95cc4a899133c320ed7d4209, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:17:55.911934 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797c9ae-d95f-4caf-a351-e52875748537 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:17:55.921463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1111/2224] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:17:55 2022] PUT /identity/v3/projects/e6e686683db641109e4b1ed65cbbd3a1/users/6dcd84cc95cc4a899133c320ed7d4209/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:17:57.607188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:57.608185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:57.608716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:57.636800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] MFA Rules not processed for user `1cc3e06e1b1440c18c0fb902beeed27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:57.638538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:57.765130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1114/2225] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:59.690773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb5076-9d74-48cb-8273-ffc21294c68c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:17:59.691311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb5076-9d74-48cb-8273-ffc21294c68c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:17:59.691455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb5076-9d74-48cb-8273-ffc21294c68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:17:59.771367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-14eb5076-9d74-48cb-8273-ffc21294c68c None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:17:59.775875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-14eb5076-9d74-48cb-8273-ffc21294c68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:17:59.909428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1112/2226] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:17:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:17:59.925114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:17:59.927276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:00.066324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bd4ea-f2e7-4ccd-b501-756ccf83fcfc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:00.067400 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bd4ea-f2e7-4ccd-b501-756ccf83fcfc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:00.067867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bd4ea-f2e7-4ccd-b501-756ccf83fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:00.075853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b81fbc-c01e-4a33-90fe-5d4c7e25c6fc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:00.082098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:00.083443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:00.083734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:00.084183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:00.108821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:00.111134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:00.117900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3c8bd4ea-f2e7-4ccd-b501-756ccf83fcfc None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:00.119349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3c8bd4ea-f2e7-4ccd-b501-756ccf83fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:00.126299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1115/2227] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:18:00.152332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1113/2228] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:00.170181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:00.179605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:00.187264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1a8ea-2754-46f9-8a11-e70c117df241 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:00.195556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:00.196229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:00.196685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:00.197749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:00.210226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:00.212460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:00.239842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1116/2229] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:18:06.491190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:06.491190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:06.491190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:06.572744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] MFA Rules not processed for user `6dcd84cc95cc4a899133c320ed7d4209`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:06.574034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:06.723537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1114/2230] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:06.824085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:06.830845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:07.011685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e03456-a04d-413b-913b-33419f25dc70 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:07.018748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142073f-a251-4434-93f4-a1431b509529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:07.019555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142073f-a251-4434-93f4-a1431b509529 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:07.019835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142073f-a251-4434-93f4-a1431b509529 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:07.020110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142073f-a251-4434-93f4-a1431b509529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:07.226208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142073f-a251-4434-93f4-a1431b509529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:07.228042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142073f-a251-4434-93f4-a1431b509529 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:07.240725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1117/2231] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:16.709219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:17.234198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=60 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:19.560591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:19.562972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:19.631024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85967c-1e6f-44b7-b24a-5909f4911563 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:19.635805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:19.636499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:19.636786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:19.637061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] PATH_INFO: `/v3/users/d542a0d1d16f466288270a8a8ee4c2b4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:19.638287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d542a0d1d16f466288270a8a8ee4c2b4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:19.640537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:19.855749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd542a0d1d16f466288270a8a8ee4c2b4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:19.952735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd542a0d1d16f466288270a8a8ee4c2b4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:19.953861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd542a0d1d16f466288270a8a8ee4c2b4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:19.954724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd542a0d1d16f466288270a8a8ee4c2b4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:19.967055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd542a0d1d16f466288270a8a8ee4c2b4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:19.994838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1115/2232] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:18:19 2022] DELETE /identity/v3/users/d542a0d1d16f466288270a8a8ee4c2b4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:20.000051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:20.000952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:20.001443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:20.042797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:20.047613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:20.130269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1118/2233] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:20.142445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:20.145080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:20.260726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae24259-b612-472c-afcc-e2a26a5f20a1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:20.265390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea4e4a-0d25-42b2-98f6-c41c003732b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:20.266126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea4e4a-0d25-42b2-98f6-c41c003732b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:20.266499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea4e4a-0d25-42b2-98f6-c41c003732b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:20.266918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea4e4a-0d25-42b2-98f6-c41c003732b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:20.278152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea4e4a-0d25-42b2-98f6-c41c003732b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:20.280358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea4e4a-0d25-42b2-98f6-c41c003732b6 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:20.290776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1116/2234] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:20.928011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:20.929706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:20.954211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4cb7a-86c0-4397-95fe-e5914ac5f381 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:20.954211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:20.954211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:20.954540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:20.954806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] PATH_INFO: `/v3/projects/1dd46de623ca4fafafe6f052c451982c` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:20.956268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd46de623ca4fafafe6f052c451982c)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:20.958460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:21.102437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd46de623ca4fafafe6f052c451982c', 'request_id': 'req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:21.111766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd46de623ca4fafafe6f052c451982c', 'request_id': 'req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:21.201296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd46de623ca4fafafe6f052c451982c', 'request_id': 'req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:21.219992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] The token cache is being invalidate because project 1dd46de623ca4fafafe6f052c451982c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:18:21.220278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:21.222148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-164ff408-f4f1-4ac2-9bb3-3854d4b4185a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:21.227582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1119/2235] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:18:20 2022] DELETE /identity/v3/projects/1dd46de623ca4fafafe6f052c451982c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:21.245751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.245751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.245751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.281479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:21.282781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:21.308620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:21.383298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1117/2236] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:21.386443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.389674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.390312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.415629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:21.417937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:21.430012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1120/2237] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:21.440224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:21.455841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804bb8-339b-4b49-a350-b6b18978990e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:21.463365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:21.464042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.464304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.464756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.465314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:18:21.465842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:21.467448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:21.487223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1118/2238] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:18:21.491324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:21.502336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c2f79-d3e0-457f-942b-b86ec811060e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:21.507643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:21.508349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.508700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.509042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.509810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:21.511849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:21.607214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1121/2239] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:18:21 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:21.611703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:21.622382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c435d-e1a1-49e0-a3e3-31da9f3deca5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:21.690829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:21.691677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.691964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.692238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.692869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:21.694781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:21.785636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1119/2240] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:18:21 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:18:21.789862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:21.806974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34375a2-9088-4a95-9a7b-9ad876e90438 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:21.816209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:21.818203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.818203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.818203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.818789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:21.821689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:21.834075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1122/2241] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:18:21.838626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:21.853600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa972c49-cef5-432e-81f6-6a279706ee79 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:21.860045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bacc7f-5a5b-4d2c-a766-6653c92292f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:21.860796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bacc7f-5a5b-4d2c-a766-6653c92292f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.861208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bacc7f-5a5b-4d2c-a766-6653c92292f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.861651 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bacc7f-5a5b-4d2c-a766-6653c92292f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.862640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bacc7f-5a5b-4d2c-a766-6653c92292f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:21.865291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bacc7f-5a5b-4d2c-a766-6653c92292f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:21.886750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1120/2242] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:18:21.891427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:21.909450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d1906-908d-45f5-b3d5-a8d1b908dc0d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:21.916419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:21.917739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:21.918029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:21.918312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] PATH_INFO: `/v3/projects/9ef16a5156d9421e85b4106cce719b4f/users/969b244842434bb09642122db0d159c0/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:21.941825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef16a5156d9421e85b4106cce719b4f, user_id=969b244842434bb09642122db0d159c0, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:21.949494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:22.067557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1123/2243] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:18:21 2022] PUT /identity/v3/projects/9ef16a5156d9421e85b4106cce719b4f/users/969b244842434bb09642122db0d159c0/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:22.072770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:22.073117 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:22.073398 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:22.100815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:22.102073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:22.187724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1121/2244] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:22.197555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:22.199446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:22.224724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:22.262002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4205-d83b-44f7-9554-55a06c81b8cd admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:22.266883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b2263-3121-4278-8b0f-78b24573f39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:22.267584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b2263-3121-4278-8b0f-78b24573f39f service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:22.267885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b2263-3121-4278-8b0f-78b24573f39f service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:22.268178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b2263-3121-4278-8b0f-78b24573f39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:22.281548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b2263-3121-4278-8b0f-78b24573f39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:22.283366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b2263-3121-4278-8b0f-78b24573f39f service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:22.301451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1124/2245] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:22.714203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:22.716383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:22.760895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139db0-870a-4355-a4f5-08e165a38ead None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:22.772456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:22.774199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:22.774890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:22.775577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] PATH_INFO: `/v3/users/fd899fb668084a39aff57b2f38498d16` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:22.778479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd899fb668084a39aff57b2f38498d16)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:22.783889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:22.849995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:23.070792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd899fb668084a39aff57b2f38498d16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:23.135064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd899fb668084a39aff57b2f38498d16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:23.135755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd899fb668084a39aff57b2f38498d16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:23.136704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd899fb668084a39aff57b2f38498d16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:23.145556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd899fb668084a39aff57b2f38498d16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:23.183252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1122/2246] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:18:22 2022] DELETE /identity/v3/users/fd899fb668084a39aff57b2f38498d16 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:23.184426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:23.185357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:23.185700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:23.215029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:23.216360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:23.356971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1125/2247] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:23.361909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:23.367793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:23.494627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a5959-0f50-4c00-b25a-d15d784e3d84 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:23.505594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ffc3c-d5a4-4cae-b2e6-04d3a7f23483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:23.506247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ffc3c-d5a4-4cae-b2e6-04d3a7f23483 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:23.506524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ffc3c-d5a4-4cae-b2e6-04d3a7f23483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:23.506796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ffc3c-d5a4-4cae-b2e6-04d3a7f23483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:23.544031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ffc3c-d5a4-4cae-b2e6-04d3a7f23483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:23.547252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ffc3c-d5a4-4cae-b2e6-04d3a7f23483 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:23.567000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1123/2248] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:25.127264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:25.130442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:25.145300 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106eb8cc-aed3-4f81-aedb-f53c532d1f28 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:25.151000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:25.151757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:25.152124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:25.152474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] PATH_INFO: `/v3/projects/eaf22fc86ccc42fb94ed40c592330133` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:25.153982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf22fc86ccc42fb94ed40c592330133)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:25.156820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:25.291009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf22fc86ccc42fb94ed40c592330133', 'request_id': 'req-42f49b6c-b452-4c4d-b76f-55e9991973c7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:25.299727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf22fc86ccc42fb94ed40c592330133', 'request_id': 'req-42f49b6c-b452-4c4d-b76f-55e9991973c7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:25.423649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf22fc86ccc42fb94ed40c592330133', 'request_id': 'req-42f49b6c-b452-4c4d-b76f-55e9991973c7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:25.438667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] The token cache is being invalidate because project eaf22fc86ccc42fb94ed40c592330133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:18:25.439377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:25.440827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-42f49b6c-b452-4c4d-b76f-55e9991973c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:25.457251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1126/2249] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:18:25 2022] DELETE /identity/v3/projects/eaf22fc86ccc42fb94ed40c592330133 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:30.253355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5d555-da88-42db-8eba-6e67c3861533 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:30.253951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5d555-da88-42db-8eba-6e67c3861533 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:30.253951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5d555-da88-42db-8eba-6e67c3861533 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:30.255881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1124/2250] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:18:30.266675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:30.267636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:30.268069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:30.320941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:30.325334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:30.434395 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:30.506591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1127/2251] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:18:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:32.587725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:32.588691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:32.589162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:32.655751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] MFA Rules not processed for user `969b244842434bb09642122db0d159c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:32.657281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:32.816099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1125/2252] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:32.817393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:32.819333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:32.922286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6f2e2-d1a3-4e43-94ff-38e19b970a07 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:32.933422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:32.934066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:32.934352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:32.934627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:32.935320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:32.938781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.038804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1128/2253] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:18:32 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:33.046774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:33.088332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fb9e7-3d6c-4763-90ee-8734a719163c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:33.164396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:33.202116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:33.202597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.202879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.203342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.203995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:33.206120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.298673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:33.300562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:33.330559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1126/2254] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:18:33 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:33.334132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:33.356155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2059b9b-9a1a-4405-ba23-aa52870c54d5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:33.376561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:33.377464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.377969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.378597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.379516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:33.382556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.413201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1127/2255] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:18:33.414724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:33.431137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bcfa3-4fba-4830-bea4-eb4f516918af admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:33.437168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:33.438269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.438830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.439357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] PATH_INFO: `/v3/projects/231c305c10ea46008717a137a8543f89/users/82daac0f2c764609810cbd188b653ccb/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.462533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258dc9-4386-42d8-8778-206eed24edac admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:33.468337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:33.469277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.469765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.470692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.476298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231c305c10ea46008717a137a8543f89, user_id=82daac0f2c764609810cbd188b653ccb, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:33.481176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.574650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:33.576462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.589677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1129/2256] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:18:33.605740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1128/2257] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:18:33 2022] PUT /identity/v3/projects/231c305c10ea46008717a137a8543f89/users/82daac0f2c764609810cbd188b653ccb/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:33.607322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:33.620350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:33.625977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672a9ca-7896-4144-a7a5-7067634e7dfe service glance] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:33.720797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d68b96-be7b-46ad-82d4-318aad1e97da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:33.722982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d68b96-be7b-46ad-82d4-318aad1e97da admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.723306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d68b96-be7b-46ad-82d4-318aad1e97da admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.724487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d68b96-be7b-46ad-82d4-318aad1e97da admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.725160 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d68b96-be7b-46ad-82d4-318aad1e97da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:33.727440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d68b96-be7b-46ad-82d4-318aad1e97da admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.742028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1130/2258] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:18:33.743407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:33.756870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d06f-cdc1-4931-9fb3-c9ac480f5275 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:33.761967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944218-e727-4f27-acad-a84add08c795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:33.762746 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944218-e727-4f27-acad-a84add08c795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.763403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944218-e727-4f27-acad-a84add08c795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.763545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944218-e727-4f27-acad-a84add08c795 admin admin] PATH_INFO: `/v3/projects/231c305c10ea46008717a137a8543f89/users/82daac0f2c764609810cbd188b653ccb/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.765597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944218-e727-4f27-acad-a84add08c795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231c305c10ea46008717a137a8543f89, user_id=82daac0f2c764609810cbd188b653ccb, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:33.768371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944218-e727-4f27-acad-a84add08c795 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:33.784712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1129/2259] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:18:33 2022] PUT /identity/v3/projects/231c305c10ea46008717a137a8543f89/users/82daac0f2c764609810cbd188b653ccb/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:33.883086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09513-f3f5-4c74-9f48-f993fbc970cd None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.883575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09513-f3f5-4c74-9f48-f993fbc970cd None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.883754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09513-f3f5-4c74-9f48-f993fbc970cd None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.885361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1131/2260] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:18:33.890435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a34e52-d9c3-4758-b75d-94df27fc0a3b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:33.890773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a34e52-d9c3-4758-b75d-94df27fc0a3b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:33.891087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a34e52-d9c3-4758-b75d-94df27fc0a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:33.917698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-86a34e52-d9c3-4758-b75d-94df27fc0a3b None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:33.920011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-86a34e52-d9c3-4758-b75d-94df27fc0a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:33.988895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1130/2261] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:34.001578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba09513-f3f5-4c74-9f48-f993fbc970cd None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:34.003778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2ba09513-f3f5-4c74-9f48-f993fbc970cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:34.022190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba09513-f3f5-4c74-9f48-f993fbc970cd None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:34.026158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f65ae5-4607-41ca-a77f-4bb7002c43d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:34.026625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f65ae5-4607-41ca-a77f-4bb7002c43d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:34.026905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f65ae5-4607-41ca-a77f-4bb7002c43d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:34.027189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f65ae5-4607-41ca-a77f-4bb7002c43d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:34.052286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f65ae5-4607-41ca-a77f-4bb7002c43d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:34.053938 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f65ae5-4607-41ca-a77f-4bb7002c43d5 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:34.065594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1132/2262] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:34.163013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756cca5-7d02-4567-84d7-bca624c54334 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:34.163358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756cca5-7d02-4567-84d7-bca624c54334 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:34.163624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756cca5-7d02-4567-84d7-bca624c54334 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:34.168305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1131/2263] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:18:34.173878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05368-b661-4f85-baf7-e3e0bed97f03 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:34.176856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05368-b661-4f85-baf7-e3e0bed97f03 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:34.176856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05368-b661-4f85-baf7-e3e0bed97f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:34.205432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f5f05368-b661-4f85-baf7-e3e0bed97f03 None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:34.207284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f5f05368-b661-4f85-baf7-e3e0bed97f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:34.364204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1133/2264] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:34.375222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756cca5-7d02-4567-84d7-bca624c54334 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:34.377201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5756cca5-7d02-4567-84d7-bca624c54334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:34.424570 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756cca5-7d02-4567-84d7-bca624c54334 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:34.439058 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:34.440477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:34.440778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:34.442143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:34.465813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:34.465813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:34.474622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1132/2265] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:34.540061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:34.540354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:34.540631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:34.569853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:34.571454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:34.642360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1134/2266] 10.222.0.62 () {58 vars in 967 bytes} [Tue Sep 13 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:34.665711 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:34.667970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:34.767060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a0a1e-6f0e-4342-b3ef-5fb94af17c1f service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:34.783342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:34.783979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:34.784260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:34.787113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:34.800752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:34.802929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:34.810147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:34.818336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1133/2267] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:18:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:18:38.444095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:38.445814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:38.507146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:38.593570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bcbab-f8c1-4eab-8e77-3320b0e78a65 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:38.604288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:38.604288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:38.604288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:38.604288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] PATH_INFO: `/v3/users/6c886285823a45a0b921a180d5bf4f1e` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:38.606148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c886285823a45a0b921a180d5bf4f1e)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:38.608199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:38.853480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c886285823a45a0b921a180d5bf4f1e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:38.960637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c886285823a45a0b921a180d5bf4f1e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:38.962314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c886285823a45a0b921a180d5bf4f1e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:38.962960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c886285823a45a0b921a180d5bf4f1e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:38.973830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bfc059f-d7a9-47d6-b8b2-30cd74a818f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c886285823a45a0b921a180d5bf4f1e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:39.017225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1135/2268] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:18:38 2022] DELETE /identity/v3/users/6c886285823a45a0b921a180d5bf4f1e => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:39.453062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:39.464562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df487760-ca55-4787-aef6-b0036a0c3545 service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:39.574899 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:39.575590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:39.575873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:39.576148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] PATH_INFO: `/v3/projects/723137b6e4764ec6b58b46f84a05a37e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:39.577885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=723137b6e4764ec6b58b46f84a05a37e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:39.580036 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:39.736954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '723137b6e4764ec6b58b46f84a05a37e', 'request_id': 'req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:39.739814 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:39.740627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:39.741074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:39.745380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '723137b6e4764ec6b58b46f84a05a37e', 'request_id': 'req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:39.764340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:39.765541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:39.837802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1136/2269] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:18:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:39.843357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '723137b6e4764ec6b58b46f84a05a37e', 'request_id': 'req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:39.855196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:39.857255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:39.872491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] The token cache is being invalidate because project 723137b6e4764ec6b58b46f84a05a37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:18:39.872932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:39.874749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7d80812e-b692-4dc1-8dd2-e2f3c9fc19bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:39.877241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1134/2270] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:18:39 2022] DELETE /identity/v3/projects/723137b6e4764ec6b58b46f84a05a37e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:39.883385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5573e71-ebd7-454b-bf71-e0b48245ed38 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:39.883796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5573e71-ebd7-454b-bf71-e0b48245ed38 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:39.884170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5573e71-ebd7-454b-bf71-e0b48245ed38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:39.921259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e5573e71-ebd7-454b-bf71-e0b48245ed38 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:39.923007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5573e71-ebd7-454b-bf71-e0b48245ed38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:39.945275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:39.968740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecb9a0-b950-4616-9107-57b4b6a5cce2 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:39.977367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:39.977997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:39.978278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:39.978550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:39.980689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:39.993595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.000576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.002561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1135/2271] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:40.011003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.013664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1137/2272] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:18:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:18:40.015427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.015571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.045046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:40.046449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:40.057149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1136/2273] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:40.063295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:40.079795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c039cc-1e01-4eba-9152-0324b1e31798 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:40.097264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:40.098821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.099225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.100243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.102328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:18:40.102746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.105111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.124504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1138/2274] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:18:40.129192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:40.144791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470200a-4a1a-4452-a2f0-783918712b18 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:40.147131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:40.147929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.148284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.148854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.149509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.151644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.276405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:40.285069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1137/2275] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:18:40 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:40.291266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370cfc-abe1-4fef-b1aa-c00e1485f91d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:40.467917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:40.468932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.469275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.469692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.470518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.473487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.583486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1139/2276] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:18:40 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:40.587138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:40.600331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:40.605756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c779c9-b96b-49a8-ba4b-67135ed04e76 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:40.620219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:40.620916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.622088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.622375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.623043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.629078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.641854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1138/2277] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:18:40.646313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:40.663490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad477-222a-4d2f-abae-cd6ec0d7bbb6 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:40.668511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3911f-6b74-41ce-a0fe-fb07d42f1c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:40.669141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3911f-6b74-41ce-a0fe-fb07d42f1c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.669409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3911f-6b74-41ce-a0fe-fb07d42f1c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.669715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3911f-6b74-41ce-a0fe-fb07d42f1c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.671129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3911f-6b74-41ce-a0fe-fb07d42f1c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.673739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3911f-6b74-41ce-a0fe-fb07d42f1c70 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.699569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1140/2278] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:18:40.700338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:40.730681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b8ed-bd46-4ae2-ad1b-b2140954426b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:40.773269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:40.773948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.774234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.774510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] PATH_INFO: `/v3/projects/9b5f1497c9de491dbb708ef6a2166f97/users/20fb1c9d1a36462d9a77318f8abdc327/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.828631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5f1497c9de491dbb708ef6a2166f97, user_id=20fb1c9d1a36462d9a77318f8abdc327, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:40.828631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:40.950662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434c61-73d0-4a6d-88f8-8c3a87b2aad3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:40.951272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434c61-73d0-4a6d-88f8-8c3a87b2aad3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:40.951533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434c61-73d0-4a6d-88f8-8c3a87b2aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:40.954365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1139/2279] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:18:40 2022] PUT /identity/v3/projects/9b5f1497c9de491dbb708ef6a2166f97/users/20fb1c9d1a36462d9a77318f8abdc327/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:41.030057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-da434c61-73d0-4a6d-88f8-8c3a87b2aad3 None None] MFA Rules not processed for user `20fb1c9d1a36462d9a77318f8abdc327`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:41.031529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-da434c61-73d0-4a6d-88f8-8c3a87b2aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:41.327576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1141/2280] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:41.335571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:41.337607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:41.389376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dbdf2-ac5f-4df8-af66-835440c2f745 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:41.390163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dbdf2-ac5f-4df8-af66-835440c2f745 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:41.390619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dbdf2-ac5f-4df8-af66-835440c2f745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:41.466414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2d0dbdf2-ac5f-4df8-af66-835440c2f745 None None] MFA Rules not processed for user `82daac0f2c764609810cbd188b653ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:41.468465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2d0dbdf2-ac5f-4df8-af66-835440c2f745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:41.499514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae7e25-8b31-4b50-8089-e56089df8c41 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:41.509107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:41.509741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:41.510131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:41.510567 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:41.512118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:41.532695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:41.534345 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:41.558903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1140/2281] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:41.671014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1142/2282] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:41.690391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:41.692747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:41.829349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb416c-9962-4cda-b9a4-def54962d0dc service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:41.843297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:41.843958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:41.844253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:41.844527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:41.846614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:41.946637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:41.951808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:41.967544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1141/2283] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:42.063726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504762a-83d4-43b6-82d5-4ed868549ebb None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:42.064635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504762a-83d4-43b6-82d5-4ed868549ebb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:42.064919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504762a-83d4-43b6-82d5-4ed868549ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:42.100004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5504762a-83d4-43b6-82d5-4ed868549ebb None None] MFA Rules not processed for user `7ddb2572cac446aa92a0bcf170d9e8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:42.101354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5504762a-83d4-43b6-82d5-4ed868549ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:42.213410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1143/2284] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:46.459777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:46.471177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:46.553083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e76596-9ddc-461b-a1d2-a6e3dd129398 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:46.557872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:46.558822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:46.559027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:46.559352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] PATH_INFO: `/v3/users/20fb1c9d1a36462d9a77318f8abdc327` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:46.560937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fb1c9d1a36462d9a77318f8abdc327)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:46.562943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:46.780045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95605cb0-83eb-4e30-bd41-6ff03cdb6623 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:46.780045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95605cb0-83eb-4e30-bd41-6ff03cdb6623 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:46.780045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95605cb0-83eb-4e30-bd41-6ff03cdb6623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:46.784676 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fb1c9d1a36462d9a77318f8abdc327'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:46.830147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-95605cb0-83eb-4e30-bd41-6ff03cdb6623 None None] MFA Rules not processed for user `969b244842434bb09642122db0d159c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:46.831487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-95605cb0-83eb-4e30-bd41-6ff03cdb6623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:46.849437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1144/2285] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:18:46.879156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fb1c9d1a36462d9a77318f8abdc327'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:46.879920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fb1c9d1a36462d9a77318f8abdc327'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:46.880540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fb1c9d1a36462d9a77318f8abdc327'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:46.890744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fb1c9d1a36462d9a77318f8abdc327'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:46.966837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1142/2286] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:18:46 2022] DELETE /identity/v3/users/20fb1c9d1a36462d9a77318f8abdc327 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:46.967843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:46.968437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:46.968982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:46.998500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:47.000025 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:47.153383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1145/2287] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:47.161718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:47.163868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:47.236469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdd73-2336-4fa6-9e88-d536d3d97580 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:47.241750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26834bad-1a14-4d96-845d-6a038af48f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:47.242509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834bad-1a14-4d96-845d-6a038af48f4c service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:47.242848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834bad-1a14-4d96-845d-6a038af48f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:47.243231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834bad-1a14-4d96-845d-6a038af48f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:47.292225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26834bad-1a14-4d96-845d-6a038af48f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:47.293686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26834bad-1a14-4d96-845d-6a038af48f4c service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:47.312318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1143/2288] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:47.940479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:47.942680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:47.962800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b82b-2c65-4c81-b038-1e3cb7d287f0 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:47.968671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:47.969581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:47.970030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:47.970614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] PATH_INFO: `/v3/projects/9b5f1497c9de491dbb708ef6a2166f97` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:47.972295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5f1497c9de491dbb708ef6a2166f97)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:47.974670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:48.133779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5f1497c9de491dbb708ef6a2166f97', 'request_id': 'req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:48.137019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:48.143876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5f1497c9de491dbb708ef6a2166f97', 'request_id': 'req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:48.569440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5f1497c9de491dbb708ef6a2166f97', 'request_id': 'req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:48.580745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] The token cache is being invalidate because project 9b5f1497c9de491dbb708ef6a2166f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:18:48.581159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:48.581879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-620c1fd0-e55d-49e5-b522-41e57c7ec6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:18:48.583757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1146/2289] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:18:47 2022] DELETE /identity/v3/projects/9b5f1497c9de491dbb708ef6a2166f97 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:48.588908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:48.590002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:48.590469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:48.613104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:48.615630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:48.643499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:48.682778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1144/2290] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:48.691279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:48.691567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:48.691867 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:48.713032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:48.714248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:48.724475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1147/2291] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:48.733191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:48.744317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e0851-cd76-4b54-a433-5f1ce114669e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:48.765338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:48.766050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:48.766328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:48.766675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:48.767442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:18:48.768162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:48.770843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:48.785525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1145/2292] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:18:48.789536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:48.800375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837042-6fcc-447b-92ea-4be7d114aed5 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:48.807227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:48.807984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:48.809435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:48.809435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:48.809435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:48.812463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:48.949828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1148/2293] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:18:48 2022] POST /identity/v3/projects => generated 391 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:48.955054 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:48.970718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4476e7-fb64-480d-bd3c-ff79ea6764a7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:49.073714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:49.075766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:49.076576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:49.076853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:49.077510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:49.079521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:49.226561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1146/2294] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:18:48 2022] POST /identity/v3/users => generated 296 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:18:49.228261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:49.241409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c56d09-c911-4ad1-a31b-afaf1483a146 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:49.249150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:49.249944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:49.250365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:49.250808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:49.251692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:49.253365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:49.266351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1149/2295] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:18:49.270715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:49.281220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374923e-aec6-4a4d-9285-a9680dfea45f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:49.286285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dbf63-8a0a-4d83-b9d1-4576ca1bb6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:49.286900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dbf63-8a0a-4d83-b9d1-4576ca1bb6da admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:49.287254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dbf63-8a0a-4d83-b9d1-4576ca1bb6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:49.287468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dbf63-8a0a-4d83-b9d1-4576ca1bb6da admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:49.288125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dbf63-8a0a-4d83-b9d1-4576ca1bb6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:49.289734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dbf63-8a0a-4d83-b9d1-4576ca1bb6da admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:49.304233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1147/2296] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:18:49.316091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:49.328709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18bfe9-2270-4a4a-bfd9-21c2d8d007f0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:49.334342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:49.335110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:49.335534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:49.336313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] PATH_INFO: `/v3/projects/fd254f919c5640269c3dcae920b8ca26/users/3a51fb7f29a441bfbf18d81c11941890/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:49.377725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd254f919c5640269c3dcae920b8ca26, user_id=3a51fb7f29a441bfbf18d81c11941890, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:49.387545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:49.596534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1150/2297] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:18:49 2022] PUT /identity/v3/projects/fd254f919c5640269c3dcae920b8ca26/users/3a51fb7f29a441bfbf18d81c11941890/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:18:49.599903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:49.600436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:49.600436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:49.629116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] MFA Rules not processed for user `3a51fb7f29a441bfbf18d81c11941890`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:18:49.630328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:49.722355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1148/2298] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:18:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:18:49.731611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:49.748704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:49.801805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bbada-11d2-41d1-94cc-c5e0f25b04f3 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:49.805787 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159f190-f63c-41a0-a0e7-ee1bf4946163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:49.806630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159f190-f63c-41a0-a0e7-ee1bf4946163 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:49.806966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159f190-f63c-41a0-a0e7-ee1bf4946163 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:49.807283 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159f190-f63c-41a0-a0e7-ee1bf4946163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:49.816550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159f190-f63c-41a0-a0e7-ee1bf4946163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:49.818119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159f190-f63c-41a0-a0e7-ee1bf4946163 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:49.827503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1151/2299] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:55.123097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:18:55.124873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:55.154245 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:18:55.195975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56acc5-6c4f-42f0-abac-9b4ac8f246bf None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:18:55.201352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:18:55.201970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:18:55.202250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:18:55.202546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:18:55.204320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:18:55.215623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:18:55.217699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:18:55.228157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1149/2300] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:18:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:18:59.823121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=58,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:19:05.247995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=21 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:19:16.577266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:16.577823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:16.577823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:16.599092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:19:16.600435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:16.657589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1152/2301] 10.222.0.62 () {58 vars in 967 bytes} [Tue Sep 13 10:19:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:19:16.683114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:19:16.685124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:16.756173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2237-9037-4ae9-82d2-50c33d0cabc7 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:19:16.761205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c22d0e-dcb2-44f5-b702-d1ef9b9fb0c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:19:16.761805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22d0e-dcb2-44f5-b702-d1ef9b9fb0c9 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:16.762081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22d0e-dcb2-44f5-b702-d1ef9b9fb0c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:16.762971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22d0e-dcb2-44f5-b702-d1ef9b9fb0c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:16.771573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c22d0e-dcb2-44f5-b702-d1ef9b9fb0c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:19:16.774598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c22d0e-dcb2-44f5-b702-d1ef9b9fb0c9 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:19:16.781818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1150/2302] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:19:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:19:18.456007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:19:18.535481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4168bc-7b71-4c3e-938b-8ede35278f18 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:19:18.539966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:19:18.540540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:18.540813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:18.541081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] PATH_INFO: `/v3/users/3a51fb7f29a441bfbf18d81c11941890` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:18.542180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a51fb7f29a441bfbf18d81c11941890)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:19:18.544665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:19:18.703094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a51fb7f29a441bfbf18d81c11941890'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:18.773414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a51fb7f29a441bfbf18d81c11941890'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:18.774118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a51fb7f29a441bfbf18d81c11941890'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:18.775309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a51fb7f29a441bfbf18d81c11941890'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:18.784893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a51fb7f29a441bfbf18d81c11941890'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:18.807322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1153/2303] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:19:18 2022] DELETE /identity/v3/users/3a51fb7f29a441bfbf18d81c11941890 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:19:18.812338 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:18.812845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:18.812930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:18.832014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:19:18.833232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:18.897443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1151/2304] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:19:18.908628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:19:18.910480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:18.942603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:19:18.992688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa68671-1133-4014-b41d-86299a005a90 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:19:18.997219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:19:18.998890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:18.999311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:18.999738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:19.015888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:19:19.017336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:19:19.044003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1154/2305] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:19:19.439844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:19:19.441380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:19.457077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78501118-67a4-46b4-83b5-2b0f78b795cd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:19:19.466255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:19:19.466255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:19.466255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:19.466255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] PATH_INFO: `/v3/projects/fd254f919c5640269c3dcae920b8ca26` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:19.466974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd254f919c5640269c3dcae920b8ca26)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:19:19.467901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:19:19.652384 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd254f919c5640269c3dcae920b8ca26', 'request_id': 'req-b6de6928-a2b2-4052-8493-bdc5075da751'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:19.659641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd254f919c5640269c3dcae920b8ca26', 'request_id': 'req-b6de6928-a2b2-4052-8493-bdc5075da751'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:19.739248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd254f919c5640269c3dcae920b8ca26', 'request_id': 'req-b6de6928-a2b2-4052-8493-bdc5075da751'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:19.751336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] The token cache is being invalidate because project fd254f919c5640269c3dcae920b8ca26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:19:19.751670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:19.752325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:19:19.753846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1152/2306] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:19:19 2022] DELETE /identity/v3/projects/fd254f919c5640269c3dcae920b8ca26 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:19:27.838027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:19:27.839657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:27.850863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c8656-b8d4-487b-80dc-e0d4fbc92a3a service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:19:27.900380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:19:27.900988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:27.901269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:27.901537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:27.904051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:27.967799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:19:27.969234 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b977fb-04cd-4eb2-b793-7f277a1d046c service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:19:27.978726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1155/2307] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:19:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:19:29.748151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:19:35.533693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:19:35.535118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:35.547176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de6928-a2b2-4052-8493-bdc5075da751 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:19:35.550750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:19:35.551375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:35.551656 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:35.551927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:35.553557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:35.628009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:19:35.628009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add38a2-d20b-4741-b8da-e6945d431ca0 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:19:35.637135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1153/2308] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:19:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:19:37.974396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=65 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:19:45.632077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=30 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:19:57.343817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986c228-f691-49b6-89da-b6a401e4bddf None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:19:57.344349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986c228-f691-49b6-89da-b6a401e4bddf None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:19:57.344349 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986c228-f691-49b6-89da-b6a401e4bddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:19:57.364130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b986c228-f691-49b6-89da-b6a401e4bddf None None] MFA Rules not processed for user `4067d48268f84854af4d672ab3f9dc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:19:57.365355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b986c228-f691-49b6-89da-b6a401e4bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:19:57.428192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1156/2309] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:19:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:20:04.643953 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:20:04.644440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:20:04.644544 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:20:04.663631 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] MFA Rules not processed for user `1cc3e06e1b1440c18c0fb902beeed27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:20:04.664831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:20:04.718287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1154/2310] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:20:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:20:07.416184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=25 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:20:14.711069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=25 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:20:22.878958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:20:22.879462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:20:22.879579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:20:22.898443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] MFA Rules not processed for user `c069999293a847cdb11650045e22576c`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:20:22.899690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:20:22.955310 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1157/2311] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:20:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:20:32.948561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=25 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:20:39.809393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:20:39.820062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab69b8a-cd40-4453-924f-d17dd77486db None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:20:39.823954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:20:39.824741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:20:39.825215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:20:39.825426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:20:39.828029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:20:39.900792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:20:39.902221 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:20:39.914449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1155/2312] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:20:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:20:46.211292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:20:46.221087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41103c08-ce42-4f8c-9186-9f0cea8d91b5 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:20:46.224679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:20:46.225291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:20:46.225572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:20:46.225846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:20:46.227603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:20:46.241199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:20:46.242603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:20:46.254314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1158/2313] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:20:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:20:49.909903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=34 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:20:53.720477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:20:53.722032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:20:53.736110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2480ef-3cac-4e24-8e44-85b1fbbccb2d service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:20:53.741639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:20:53.742247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:20:53.742556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:20:53.742827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:20:53.744578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:20:53.755619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:20:53.757190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:20:53.766161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1156/2314] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:20:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:20:56.249954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=6 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:01.656029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:01.667693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d785b7e-8b5e-4448-8acd-143948e04400 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:01.673997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:01.674604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:01.674888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:01.675178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:01.677020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:01.687672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:01.689259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:01.698460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1159/2315] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:03.761888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:11.149179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:11.149179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:11.152838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decb32c-90ba-49b7-92f7-d7566bcee2da service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:11.158484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:11.159320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:11.159696 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:11.160066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:11.162080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:11.183148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:11.185197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:11.196984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1157/2316] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:11.694743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=6 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:21.191997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:21.679239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:21.689994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e681-d8a0-409e-b6b5-acfdc0dc1c38 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:21.694833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:21.695749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:21.696173 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:21.696749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:21.698898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:21.762117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:21.763633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:21.776604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1160/2317] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:21:25.796431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:25.811024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224484d-4b97-4de8-a7be-50460b13b21c service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:25.819261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:25.821880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:25.821880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:25.821880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:25.826832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:25.849903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:25.852253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:25.864856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1158/2318] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:26.066241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:26.068128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:26.087308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7844cee-fcb9-472b-8023-9434e20fe8f4 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:26.088652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:26.089296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:26.089604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:26.089876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] PATH_INFO: `/v3/users/1cc3e06e1b1440c18c0fb902beeed27d` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:26.091042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc3e06e1b1440c18c0fb902beeed27d)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:26.093029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:26.328738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc3e06e1b1440c18c0fb902beeed27d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:26.405311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc3e06e1b1440c18c0fb902beeed27d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:26.408763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc3e06e1b1440c18c0fb902beeed27d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:26.408763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc3e06e1b1440c18c0fb902beeed27d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:26.423011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0c09b74b-c55a-4013-8af0-a44fa8479564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc3e06e1b1440c18c0fb902beeed27d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:26.446495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1161/2319] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:21:26 2022] DELETE /identity/v3/users/1cc3e06e1b1440c18c0fb902beeed27d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:26.722718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:26.733621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65b4d2-ad77-43b7-a59c-0d4613893cf1 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:26.800127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:26.800729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:26.800987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:26.801255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] PATH_INFO: `/v3/projects/602cd87d731241e3ad12032e2dd67c1b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:26.802368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=602cd87d731241e3ad12032e2dd67c1b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:26.804457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:26.925721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '602cd87d731241e3ad12032e2dd67c1b', 'request_id': 'req-5c911892-7373-4ff6-b243-461ee042e785'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:26.932013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '602cd87d731241e3ad12032e2dd67c1b', 'request_id': 'req-5c911892-7373-4ff6-b243-461ee042e785'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:27.001380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '602cd87d731241e3ad12032e2dd67c1b', 'request_id': 'req-5c911892-7373-4ff6-b243-461ee042e785'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:27.013540 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] The token cache is being invalidate because project 602cd87d731241e3ad12032e2dd67c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:21:27.015926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:27.015926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5c911892-7373-4ff6-b243-461ee042e785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:27.016332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1159/2320] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:21:26 2022] DELETE /identity/v3/projects/602cd87d731241e3ad12032e2dd67c1b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:27.021359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.021895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.022254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.043144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:27.044398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:27.104709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1162/2321] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:27.113232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.113525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.114265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.134434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:27.135736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:27.144470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1160/2322] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:27.148901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.160307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b624d7-caec-45c2-811f-37dfb76f6e83 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.165055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.166228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.166228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.166228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.166873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:21:27.167215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.168816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.171066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:27.182335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1163/2323] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:21:27.187042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.196925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d8dc6-f2fb-4cfd-91f8-fc49c0400c53 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.202020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaeb12-0a84-47e8-8cf9-68b289721687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.202628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaeb12-0a84-47e8-8cf9-68b289721687 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.202915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaeb12-0a84-47e8-8cf9-68b289721687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.203218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaeb12-0a84-47e8-8cf9-68b289721687 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.203890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaeb12-0a84-47e8-8cf9-68b289721687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.205928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaeb12-0a84-47e8-8cf9-68b289721687 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.321055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1161/2324] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:27.326621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.337658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a6aa3-26df-4983-b3ab-fe11d45b653e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.395389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.396009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.396294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.396570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.397205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.399141 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.408895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.409716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.410176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.429001 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] MFA Rules not processed for user `4067d48268f84854af4d672ab3f9dc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:27.430261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:27.475015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:27.479807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1164/2325] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:27.481596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.487950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1162/2326] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:21:27.507140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57997015-ba04-4e5c-903c-5a1b0b986493 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.514419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.515316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.520548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.520548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.520548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.520548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.540191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1165/2327] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:27.540328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.547757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a036-232a-4f1d-86e4-566919db1b28 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.553735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e1b0-3551-484f-9fb6-6e35f7aeb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.554401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e1b0-3551-484f-9fb6-6e35f7aeb019 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.554717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e1b0-3551-484f-9fb6-6e35f7aeb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.554990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e1b0-3551-484f-9fb6-6e35f7aeb019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.555672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e1b0-3551-484f-9fb6-6e35f7aeb019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.557439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e1b0-3551-484f-9fb6-6e35f7aeb019 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.572708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1163/2328] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:27.574232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.584490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b134377-5c53-490f-aadb-7a1a8586689e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.589434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.590115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.590393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.590672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] PATH_INFO: `/v3/projects/c9af9fc38bf64f46b9aef7f0a9a702c7/users/0e9465d8b9f14cc0b01759980489920b/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.605933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9af9fc38bf64f46b9aef7f0a9a702c7, user_id=0e9465d8b9f14cc0b01759980489920b, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.608824 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.667435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1166/2329] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:27 2022] PUT /identity/v3/projects/c9af9fc38bf64f46b9aef7f0a9a702c7/users/0e9465d8b9f14cc0b01759980489920b/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:21:27.684902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.685185 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.685457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.713783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] MFA Rules not processed for user `0e9465d8b9f14cc0b01759980489920b`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:27.715043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:27.784859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1164/2330] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:27.789261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.816627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.818159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:27.853747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe83e91-58e3-475a-a3bb-6e5858d4d5f2 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.858383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.858985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.859297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.859567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.860201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.863148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.877246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d5c2a-7aff-4fbe-a6b8-627f8cec70df None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:27.881403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:27.882074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:27.882354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:27.882646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:27.884409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:27.894426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:27.895877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:27.904192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1165/2331] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:21:27.972389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1167/2332] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:27.976409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:27.988818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80aa76-102f-4d75-9559-7715f93b0900 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.046980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.047717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.048003 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.048272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.048913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.050922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.175781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1166/2333] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:21:27 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:21:28.179794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:28.190257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79607b-cb3e-4e26-9b46-d43530161788 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.194948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.195584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.195866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.196139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.196778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.198383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.209911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1168/2334] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:28.214146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:28.220752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:28.225270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc582fb-beab-49cb-95fc-1ba1cb400f6a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.229957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.230604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.230887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.231179 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] PATH_INFO: `/v3/projects/d2d81f9bad8842df8ba851121868b64a/users/93619567f56b4db18d21a15f04bf0158/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.246257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d81f9bad8842df8ba851121868b64a, user_id=93619567f56b4db18d21a15f04bf0158, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.249115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.320329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1167/2335] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:28 2022] PUT /identity/v3/projects/d2d81f9bad8842df8ba851121868b64a/users/93619567f56b4db18d21a15f04bf0158/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:28.322641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:28.331653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:28.333457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:28.334694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777efba7-bd8e-4bc8-8d6f-31c8146bbe69 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.355792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:28.408361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.408951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.409253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.409522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.410161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.411896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.421536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1169/2336] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:28.425685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:28.435256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d89b-23e0-4d8a-a04f-1f80d5fd49c6 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.437244 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3164-f39a-4e64-ae72-cc6d5f375de0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.439998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41d991-71c7-455c-8184-b4dffc2ff78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.441180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41d991-71c7-455c-8184-b4dffc2ff78b admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.441463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41d991-71c7-455c-8184-b4dffc2ff78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.441739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41d991-71c7-455c-8184-b4dffc2ff78b admin admin] PATH_INFO: `/v3/projects/d2d81f9bad8842df8ba851121868b64a/users/93619567f56b4db18d21a15f04bf0158/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.443447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.443677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41d991-71c7-455c-8184-b4dffc2ff78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d81f9bad8842df8ba851121868b64a, user_id=93619567f56b4db18d21a15f04bf0158, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.444192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.447211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.447211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.447436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41d991-71c7-455c-8184-b4dffc2ff78b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.447527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:28.458044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1170/2337] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:28 2022] PUT /identity/v3/projects/d2d81f9bad8842df8ba851121868b64a/users/93619567f56b4db18d21a15f04bf0158/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:21:28.462802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.462802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.471859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.472269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.472640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.473795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1168/2338] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:21:28.507709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] MFA Rules not processed for user `93619567f56b4db18d21a15f04bf0158`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:28.508951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:28.587183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1171/2339] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:21:28.601463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:28.603114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:28.660254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f5fd9-a878-409c-b859-8974b758ec38 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:28.709080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b96da-7e47-4010-b60f-79b0bcffd458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:28.709671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b96da-7e47-4010-b60f-79b0bcffd458 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:28.709955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b96da-7e47-4010-b60f-79b0bcffd458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:28.710228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b96da-7e47-4010-b60f-79b0bcffd458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:28.719477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b96da-7e47-4010-b60f-79b0bcffd458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:28.721069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b96da-7e47-4010-b60f-79b0bcffd458 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:28.731061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1169/2340] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:29.836090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:29.901075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48658-8756-4edc-9fb5-839a61839937 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:29.906176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:29.908031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:29.908031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:29.908031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] PATH_INFO: `/v3/users/0e9465d8b9f14cc0b01759980489920b` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:29.908754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e9465d8b9f14cc0b01759980489920b)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:29.910759 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:30.066448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:30.079026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e9465d8b9f14cc0b01759980489920b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:30.169855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e9465d8b9f14cc0b01759980489920b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:30.170512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e9465d8b9f14cc0b01759980489920b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:30.171126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e9465d8b9f14cc0b01759980489920b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:30.179783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e9465d8b9f14cc0b01759980489920b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:30.198057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1172/2341] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:21:29 2022] DELETE /identity/v3/users/0e9465d8b9f14cc0b01759980489920b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:30.202777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:30.203313 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:30.203521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:30.222995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:30.224265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:30.249546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:30.285190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1170/2342] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:30.295430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:30.296907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:30.354835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28560255-720c-4124-98cd-da0486bf7d38 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:30.358582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:30.359249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:30.359530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:30.359797 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:30.369933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:30.371189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:30.385465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1173/2343] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:30.874359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:30.874359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:30.896356 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ef554-72b6-4e11-b362-c64e1b505567 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:30.901171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:30.902679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:30.902679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:30.902679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] PATH_INFO: `/v3/projects/c9af9fc38bf64f46b9aef7f0a9a702c7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:30.903757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9af9fc38bf64f46b9aef7f0a9a702c7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:30.905681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:31.054551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9af9fc38bf64f46b9aef7f0a9a702c7', 'request_id': 'req-8d13340c-8b7d-44dd-b19e-a9abb33c8146'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.060927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9af9fc38bf64f46b9aef7f0a9a702c7', 'request_id': 'req-8d13340c-8b7d-44dd-b19e-a9abb33c8146'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.129905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9af9fc38bf64f46b9aef7f0a9a702c7', 'request_id': 'req-8d13340c-8b7d-44dd-b19e-a9abb33c8146'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.142364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] The token cache is being invalidate because project c9af9fc38bf64f46b9aef7f0a9a702c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:21:31.142896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.144077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.145661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1171/2344] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:21:30 2022] DELETE /identity/v3/projects/c9af9fc38bf64f46b9aef7f0a9a702c7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:31.149678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:31.160639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fefa9-1a1a-41a1-81b5-d84301c0df0e service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:31.218379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:31.219053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:31.219366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:31.219639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] PATH_INFO: `/v3/users/93619567f56b4db18d21a15f04bf0158` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:31.220770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93619567f56b4db18d21a15f04bf0158)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:31.223508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:31.444489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93619567f56b4db18d21a15f04bf0158'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.512926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93619567f56b4db18d21a15f04bf0158'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.515788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93619567f56b4db18d21a15f04bf0158'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.516507 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93619567f56b4db18d21a15f04bf0158'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.519692 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:31.527233 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93619567f56b4db18d21a15f04bf0158'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:31.545832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1174/2345] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:21:31 2022] DELETE /identity/v3/users/93619567f56b4db18d21a15f04bf0158 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:31.564005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:31.566119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:31.637368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13340c-8b7d-44dd-b19e-a9abb33c8146 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:31.693747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:31.694351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:31.694626 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:31.694894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:31.696682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:31.706639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:31.708170 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd726993-3e9c-4ce0-938b-ce25fb6e18b7 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:31.716763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1172/2346] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:32.035082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.036782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:32.107643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b096aa-082b-49ff-b9c8-1b0bbf77e1b7 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.112857 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.113535 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.113771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.114046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] PATH_INFO: `/v3/projects/d2d81f9bad8842df8ba851121868b64a` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.115222 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d81f9bad8842df8ba851121868b64a)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.117148 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:32.251586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d81f9bad8842df8ba851121868b64a', 'request_id': 'req-89d240c0-4ed4-427a-bc62-4e058c9b17a9'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:32.257811 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d81f9bad8842df8ba851121868b64a', 'request_id': 'req-89d240c0-4ed4-427a-bc62-4e058c9b17a9'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:32.335690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d81f9bad8842df8ba851121868b64a', 'request_id': 'req-89d240c0-4ed4-427a-bc62-4e058c9b17a9'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:32.347934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] The token cache is being invalidate because project d2d81f9bad8842df8ba851121868b64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:21:32.348226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:32.348879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89d240c0-4ed4-427a-bc62-4e058c9b17a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:32.350480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1175/2347] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:21:32 2022] DELETE /identity/v3/projects/d2d81f9bad8842df8ba851121868b64a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:32.355424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.355823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.356171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.376634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:32.378025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:32.384350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:32.440098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1173/2348] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:32.447798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.448097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.448353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.468066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:32.469274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:32.478738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1176/2349] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:32.483141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.496418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d67ec-260c-481e-9239-ea775e641d89 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.499205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.499821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.500098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.500367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.501009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:21:32.501307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.502892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:32.516157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1174/2350] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:21:32.521975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.533230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e97080-3092-48e3-b1d9-3527e377f9b2 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.542773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.542773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.542773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.542773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.543417 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.545756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:32.632110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1177/2351] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:21:32 2022] POST /identity/v3/projects => generated 357 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:32.636035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.648425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daffd12-b866-49bd-8360-a20ea778363a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.715913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.716627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.717062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.717440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.718177 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.720207 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:32.798455 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1175/2352] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:21:32 2022] POST /identity/v3/users => generated 279 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:32.802618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.813140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c12efe-e2a4-4c16-a3e8-ab6f07d6b94d admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.818065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.818748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.819125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.819542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.820198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.821823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:32.832866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1178/2353] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:32.840174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.851380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1652d58-7f26-4f56-982a-f8c4c946f875 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.856285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14a58-0881-4fc0-9f73-ad1384f7c194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.856895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14a58-0881-4fc0-9f73-ad1384f7c194 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.857229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14a58-0881-4fc0-9f73-ad1384f7c194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.857502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14a58-0881-4fc0-9f73-ad1384f7c194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.858163 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14a58-0881-4fc0-9f73-ad1384f7c194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.859778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14a58-0881-4fc0-9f73-ad1384f7c194 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:32.875376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1176/2354] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:32.882185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:32.899993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f51f6-9064-45f3-ac5b-f24a6f23718e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:32.906047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:32.906850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:32.907364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:32.908597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] PATH_INFO: `/v3/projects/da1a0bd278ef4305b698810d7bec701a/users/f93e60ec8cde43798fb84c62fa51ebb5/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:32.928221 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1a0bd278ef4305b698810d7bec701a, user_id=f93e60ec8cde43798fb84c62fa51ebb5, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:32.931846 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.002063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:33.010165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1179/2355] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:32 2022] PUT /identity/v3/projects/da1a0bd278ef4305b698810d7bec701a/users/f93e60ec8cde43798fb84c62fa51ebb5/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:33.022605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.022986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.023447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.051926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] MFA Rules not processed for user `f93e60ec8cde43798fb84c62fa51ebb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:33.053210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:33.119321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1177/2356] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:21:33.123830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.188145 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2939e02-f32b-4c46-8787-bd985d6fce21 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.192826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.193426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.193700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.193967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.194603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.196553 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.308427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1180/2357] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:21:33 2022] POST /identity/v3/projects => generated 357 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:21:33.312614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.323671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c7e07-1744-450c-ada7-d5acec31b893 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.327411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:33.379055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.379717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.379959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.380237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.380869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.383240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.477133 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1178/2358] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:21:33 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:33.479770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.490501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec77b4e-f3de-435e-b652-e5bce604ee52 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.495351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.495951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.496268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.496545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.497185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.498779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.509256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1181/2359] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:33.516219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.526571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649624-3525-4272-9437-5191d1da99dc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.531492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.532135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.532416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.532692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] PATH_INFO: `/v3/projects/25ef53cb171a4f25862ffe4ccbbd01a5/users/adbc97851a6340f0aa98812b0031b895/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.547770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ef53cb171a4f25862ffe4ccbbd01a5, user_id=adbc97851a6340f0aa98812b0031b895, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.550561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.622862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1179/2360] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:33 2022] PUT /identity/v3/projects/25ef53cb171a4f25862ffe4ccbbd01a5/users/adbc97851a6340f0aa98812b0031b895/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:33.626815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.637691 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3e2a7-b1e2-4836-af05-c7a195ea3c63 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.693400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d626006-5a8e-4cbf-8799-2d439a4f59aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.693995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d626006-5a8e-4cbf-8799-2d439a4f59aa admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.694401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d626006-5a8e-4cbf-8799-2d439a4f59aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.694675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d626006-5a8e-4cbf-8799-2d439a4f59aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.695333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d626006-5a8e-4cbf-8799-2d439a4f59aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.696924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d626006-5a8e-4cbf-8799-2d439a4f59aa admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.707009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1182/2361] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:33.711121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.721299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424c8c-1a74-4f91-b0b7-7105323ebc84 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.725944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.726589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.726864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.727213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] PATH_INFO: `/v3/projects/25ef53cb171a4f25862ffe4ccbbd01a5/users/adbc97851a6340f0aa98812b0031b895/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.729059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ef53cb171a4f25862ffe4ccbbd01a5, user_id=adbc97851a6340f0aa98812b0031b895, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.731741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.742053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1180/2362] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:33 2022] PUT /identity/v3/projects/25ef53cb171a4f25862ffe4ccbbd01a5/users/adbc97851a6340f0aa98812b0031b895/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:33.752367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.753165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.753446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.781817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] MFA Rules not processed for user `adbc97851a6340f0aa98812b0031b895`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:33.783066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:33.842433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:33.852477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1183/2363] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:33.864495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:33.866290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:33.926592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fcda-3519-461b-987b-65d7be8f98f3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:33.930334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:33.930938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:33.931248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:33.931523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:33.940919 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:33.942331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:33.951797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1181/2364] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:34.163687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:34.173428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c396d3-df03-48bb-ab9e-48763e5a4e0a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:34.177476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:34.178083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:34.178370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:34.178643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:34.234370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:34.235868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:34.245225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1184/2365] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:38.923035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:38.924621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:39.018712 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:39.036161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4ce17-22f1-46cd-87d1-1af1353d1182 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:39.043347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:39.043994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:39.044560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:39.044869 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:39.047005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:39.060619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:39.062790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c53d-1c98-479d-8bdc-c05c696ff259 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:39.075063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1182/2366] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:43.677951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:43.679552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:43.697029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14729ed1-760e-4b57-b3bc-77438b3a62bb service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:43.702962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:43.703707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:43.704084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:43.704428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:43.706809 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:43.719789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:43.721474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b239d-273b-46f3-8490-e8e50fc18888 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:43.729253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:43.729253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:43.729792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:43.734139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1185/2367] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:43.749796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] MFA Rules not processed for user `90f07b101240464387f3fbf92c925ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:43.751016 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:43.819424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1183/2368] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:47.701307 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:47.701832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:47.702152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:47.727311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:47.728716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:47.787935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1186/2369] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:47.960605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:47.962430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:48.053643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6378dd0-ed4f-4a95-b2bf-7b9ef103b69d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:48.059375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a817b06-54c8-4fac-874f-ab1d3ae80781 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:48.060033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817b06-54c8-4fac-874f-ab1d3ae80781 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:48.060318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817b06-54c8-4fac-874f-ab1d3ae80781 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:48.060594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817b06-54c8-4fac-874f-ab1d3ae80781 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:48.070528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a817b06-54c8-4fac-874f-ab1d3ae80781 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:48.072189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a817b06-54c8-4fac-874f-ab1d3ae80781 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:48.080920 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1184/2370] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:21:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:21:53.895897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:53.989686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e5e8c-d96e-4512-adec-ec8cf52d3fa7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:53.999366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:54.000313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:54.002643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:54.002953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] PATH_INFO: `/v3/users/f93e60ec8cde43798fb84c62fa51ebb5` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:54.008576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93e60ec8cde43798fb84c62fa51ebb5)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:54.011631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:54.107439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:54.210844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93e60ec8cde43798fb84c62fa51ebb5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:54.280639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93e60ec8cde43798fb84c62fa51ebb5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:54.281317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93e60ec8cde43798fb84c62fa51ebb5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:54.281937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93e60ec8cde43798fb84c62fa51ebb5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:54.317521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ad02ce23-926f-403d-86af-a1303cf32163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93e60ec8cde43798fb84c62fa51ebb5'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:54.354192 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1187/2371] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:21:53 2022] DELETE /identity/v3/users/f93e60ec8cde43798fb84c62fa51ebb5 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:54.356667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:54.357328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:54.357328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:54.385420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:54.388131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:54.485945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:54.486245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:54.486529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:54.513012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] MFA Rules not processed for user `7ddb2572cac446aa92a0bcf170d9e8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:54.514781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:54.515745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1185/2372] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:54.529726 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:54.531859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:54.569721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:54.591020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1188/2373] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:54.642941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343199bb-8ca0-45e2-ba92-0daefb28fb8a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:54.647639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:54.648324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:54.653434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:54.653434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:54.661523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:54.663243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:54.687028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1186/2374] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:55.135008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:55.141467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:55.165784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceded5-c714-4383-927e-6805e6becf81 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:55.178912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:55.179882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:55.180403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:55.180778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:55.196155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:55.214122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:55.216873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:55.230816 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1189/2375] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:21:55.278774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:55.278774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:55.293223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7e7c-8261-4659-b8b3-5b6158712405 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:55.298862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:55.299599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:55.300704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:55.301013 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] PATH_INFO: `/v3/projects/da1a0bd278ef4305b698810d7bec701a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:55.302257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1a0bd278ef4305b698810d7bec701a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:55.304635 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:55.479430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1a0bd278ef4305b698810d7bec701a', 'request_id': 'req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:55.488254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1a0bd278ef4305b698810d7bec701a', 'request_id': 'req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:55.598660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1a0bd278ef4305b698810d7bec701a', 'request_id': 'req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:55.626189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] The token cache is being invalidate because project da1a0bd278ef4305b698810d7bec701a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:21:55.626843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:55.632664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:55.638347 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1187/2376] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:21:55 2022] DELETE /identity/v3/projects/da1a0bd278ef4305b698810d7bec701a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:55.640041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:55.665538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e0df-9761-4d4a-ab43-5874f07b67a7 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:55.764762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:55.765154 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:55.765325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:55.765723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] PATH_INFO: `/v3/users/adbc97851a6340f0aa98812b0031b895` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:55.769471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adbc97851a6340f0aa98812b0031b895)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:55.769850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:56.164621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:56.178047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adbc97851a6340f0aa98812b0031b895'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:56.263634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adbc97851a6340f0aa98812b0031b895'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:56.264357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adbc97851a6340f0aa98812b0031b895'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:56.265471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adbc97851a6340f0aa98812b0031b895'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:56.279020 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fe933643-f269-4438-bb16-e456ff9cbdd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adbc97851a6340f0aa98812b0031b895'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:56.332150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1190/2377] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:21:55 2022] DELETE /identity/v3/users/adbc97851a6340f0aa98812b0031b895 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:57.377530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:57.381979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:57.403682 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb32d5-02d7-4593-a568-8e5493c9eb4b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:57.509208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:57.510040 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:57.510426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:57.510805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] PATH_INFO: `/v3/projects/25ef53cb171a4f25862ffe4ccbbd01a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:57.512124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ef53cb171a4f25862ffe4ccbbd01a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:57.516273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:57.658268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ef53cb171a4f25862ffe4ccbbd01a5', 'request_id': 'req-53480f6e-0214-4482-8092-c4065581fb97'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:57.659902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:57.665895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ef53cb171a4f25862ffe4ccbbd01a5', 'request_id': 'req-53480f6e-0214-4482-8092-c4065581fb97'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:57.740513 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ef53cb171a4f25862ffe4ccbbd01a5', 'request_id': 'req-53480f6e-0214-4482-8092-c4065581fb97'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:57.759569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] The token cache is being invalidate because project 25ef53cb171a4f25862ffe4ccbbd01a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:21:57.759883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:57.760543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-53480f6e-0214-4482-8092-c4065581fb97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:21:57.762342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1188/2378] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:21:57 2022] DELETE /identity/v3/projects/25ef53cb171a4f25862ffe4ccbbd01a5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:57.770459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:57.770983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:57.771377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:57.809865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:57.811967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:57.905083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1191/2379] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:57.914654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:57.915060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:57.915362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:57.938527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:57.943422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:57.971662 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:57.977554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1189/2380] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:57.984359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e06db9-4cb2-4a97-b4c2-5708b907a706 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:57.994604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:57.995539 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:57.995994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:57.996419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:57.997257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:21:57.997796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:57.999532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.017812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1192/2381] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:21:58.024308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.033522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5df35-3833-431a-bd3a-36ccae686b4c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:58.040763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:58.040763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.040763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.040763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.041124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:58.043267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.129561 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1190/2382] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:21:58 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:21:58.133520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.151412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde1ba-68b3-4734-9fc3-aebb716c9f74 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:58.277558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:58.278268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.278611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.278944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.279686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:58.281697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.354302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1193/2383] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:21:58 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:58.355681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.367090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846f63a-db63-426a-b4cb-849c42cda24c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:58.371995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:58.372687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.373112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.373417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.374376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:58.376089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.390580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1191/2384] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:58.393342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.406858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf4406-405d-42d0-bb27-5a79a98d16bd admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:58.414690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04368736-76dc-43e6-b1bc-f681a5a08555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:58.415414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04368736-76dc-43e6-b1bc-f681a5a08555 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.415835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04368736-76dc-43e6-b1bc-f681a5a08555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.416413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04368736-76dc-43e6-b1bc-f681a5a08555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.417136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04368736-76dc-43e6-b1bc-f681a5a08555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:58.418804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04368736-76dc-43e6-b1bc-f681a5a08555 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.431451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1194/2385] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:58.436115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.448022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb2854-9ef7-48d2-baa5-8b9fad17f81e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:58.455204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:58.455204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.455204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.455204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] PATH_INFO: `/v3/projects/98305b4eeb9b4e9ca5102effb3d1fbd1/users/df0a92dc56c6442ab5b8551bf6c1f58a/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.479343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=98305b4eeb9b4e9ca5102effb3d1fbd1, user_id=df0a92dc56c6442ab5b8551bf6c1f58a, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:58.480702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.608869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.609170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.609443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.610477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1192/2386] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:58 2022] PUT /identity/v3/projects/98305b4eeb9b4e9ca5102effb3d1fbd1/users/df0a92dc56c6442ab5b8551bf6c1f58a/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:58.653707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] MFA Rules not processed for user `df0a92dc56c6442ab5b8551bf6c1f58a`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:58.655413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:58.665659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:58.741436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1195/2387] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:21:58.745913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.841647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84c8fc-0e17-4e07-a254-60d3647753ec admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:58.853231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:58.853862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:58.856053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:58.856336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:58.857005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:58.859206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:58.959482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1193/2388] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:21:58 2022] POST /identity/v3/projects => generated 367 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:21:58.963640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:58.978273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70c3d1-55c7-4eaa-81b0-9505e7e9b6ee None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:59.052199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:59.052944 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.053310 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.053650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.054402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:59.056585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:59.199031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1196/2389] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:21:58 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:21:59.203621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:59.221939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0fa7-ac73-4811-b36f-514f3a7cb3ed admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:59.227835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:59.228963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.229249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.229602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.230253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:59.232714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:59.248565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1194/2390] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:59.253963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:59.285072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86bde5-1fdf-4190-bf38-8cdfd41e6537 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:59.301643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:59.302530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.302820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.303379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] PATH_INFO: `/v3/projects/8f086233604a49498a6709a26a89b452/users/4d5286924bc440ccb88892b6d14f1608/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.328095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f086233604a49498a6709a26a89b452, user_id=4d5286924bc440ccb88892b6d14f1608, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:59.328095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:59.418473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1197/2391] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:59 2022] PUT /identity/v3/projects/8f086233604a49498a6709a26a89b452/users/4d5286924bc440ccb88892b6d14f1608/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:59.424315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:59.445799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750a7b7-8ae1-448b-9900-29077d25cb9c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:59.497274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:21:59.513075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf0fde-1432-41be-b877-3ad4c2807fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:59.513695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf0fde-1432-41be-b877-3ad4c2807fab admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.513978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf0fde-1432-41be-b877-3ad4c2807fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.514256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf0fde-1432-41be-b877-3ad4c2807fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.514902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf0fde-1432-41be-b877-3ad4c2807fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:59.516664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf0fde-1432-41be-b877-3ad4c2807fab admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:59.526423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1195/2392] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:21:59.531295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:59.542129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2b21-8d7d-4880-94cd-010d664fd7ba admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:59.546742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:59.547430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.547711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.547987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] PATH_INFO: `/v3/projects/8f086233604a49498a6709a26a89b452/users/4d5286924bc440ccb88892b6d14f1608/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.550402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f086233604a49498a6709a26a89b452, user_id=4d5286924bc440ccb88892b6d14f1608, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:59.553090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:59.564433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1198/2393] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:21:59 2022] PUT /identity/v3/projects/8f086233604a49498a6709a26a89b452/users/4d5286924bc440ccb88892b6d14f1608/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:21:59.581952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.582237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.582510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.610063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] MFA Rules not processed for user `4d5286924bc440ccb88892b6d14f1608`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:21:59.611838 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:59.679708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1196/2394] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:21:59.691593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:21:59.693552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:21:59.759839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfb210-1e45-4a5f-ae8b-4e1892a79b91 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:21:59.763953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:21:59.764656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:21:59.764939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:21:59.765214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:21:59.775236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:21:59.776728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:21:59.787363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1199/2395] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:21:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:00.100187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:00.111716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029851a-6093-4701-b8e5-600b8f8d544b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:00.115876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9db9b5-0c45-4700-81c5-9af1074717cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:00.116634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db9b5-0c45-4700-81c5-9af1074717cd service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:00.117057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db9b5-0c45-4700-81c5-9af1074717cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:00.117456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db9b5-0c45-4700-81c5-9af1074717cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:00.176980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9db9b5-0c45-4700-81c5-9af1074717cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:00.178409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9db9b5-0c45-4700-81c5-9af1074717cd service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:00.187525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1197/2396] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:07.135738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:07.141064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:07.202718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad4e7-cef3-4b12-90d9-2767451a11eb service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:07.207775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:07.208415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:07.208699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:07.208978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] PATH_INFO: `/v3/users/df0a92dc56c6442ab5b8551bf6c1f58a` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:07.210113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0a92dc56c6442ab5b8551bf6c1f58a)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:07.212154 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:07.430378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0a92dc56c6442ab5b8551bf6c1f58a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:07.520084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0a92dc56c6442ab5b8551bf6c1f58a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:07.535334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0a92dc56c6442ab5b8551bf6c1f58a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:07.535334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0a92dc56c6442ab5b8551bf6c1f58a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:07.535334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0a92dc56c6442ab5b8551bf6c1f58a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:07.556277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1200/2397] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:07 2022] DELETE /identity/v3/users/df0a92dc56c6442ab5b8551bf6c1f58a => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:07.558093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:07.558539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:07.558633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:07.578397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:07.579693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:07.638713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1198/2398] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:07.648710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:07.650191 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:07.709028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a26e-c465-4c65-ad58-d3fffcd4c7f1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:07.712635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:07.713240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:07.713523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:07.713798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:07.723328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:07.724853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:07.734335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1201/2399] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:08.235738 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:08.237202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:08.249103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06efa33-ca11-4d93-96e9-a8135711e3aa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:08.254190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:08.254826 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:08.255109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:08.255628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] PATH_INFO: `/v3/projects/98305b4eeb9b4e9ca5102effb3d1fbd1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:08.256862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98305b4eeb9b4e9ca5102effb3d1fbd1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:08.258774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:08.261328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:08.427581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98305b4eeb9b4e9ca5102effb3d1fbd1', 'request_id': 'req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.438006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98305b4eeb9b4e9ca5102effb3d1fbd1', 'request_id': 'req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.566085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98305b4eeb9b4e9ca5102effb3d1fbd1', 'request_id': 'req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.585991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] The token cache is being invalidate because project 98305b4eeb9b4e9ca5102effb3d1fbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:08.586371 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.587043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.589000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1199/2400] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:08 2022] DELETE /identity/v3/projects/98305b4eeb9b4e9ca5102effb3d1fbd1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:08.592970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:08.602922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02747b0-356c-4b32-9d6a-503ff6012b20 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:08.629937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:08.662324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:08.663116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:08.664844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:08.664844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] PATH_INFO: `/v3/users/4d5286924bc440ccb88892b6d14f1608` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:08.664844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5286924bc440ccb88892b6d14f1608)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:08.668093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:08.898591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5286924bc440ccb88892b6d14f1608'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.959600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5286924bc440ccb88892b6d14f1608'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.960261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5286924bc440ccb88892b6d14f1608'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.960892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5286924bc440ccb88892b6d14f1608'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.968529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-79945e73-7d19-4594-8b71-14a2f525fc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5286924bc440ccb88892b6d14f1608'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:08.988198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1202/2401] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:08 2022] DELETE /identity/v3/users/4d5286924bc440ccb88892b6d14f1608 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:09.468621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:09.470954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:09.493298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb36ac4-8e7d-4bd9-bdc7-bc8b21789c8c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:09.571119 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:09.571893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:09.575592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:09.575592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] PATH_INFO: `/v3/projects/8f086233604a49498a6709a26a89b452` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:09.575592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f086233604a49498a6709a26a89b452)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:09.576449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:09.725739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f086233604a49498a6709a26a89b452', 'request_id': 'req-98022223-98f8-46e4-83cf-4c5fc37f07ff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:09.733077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f086233604a49498a6709a26a89b452', 'request_id': 'req-98022223-98f8-46e4-83cf-4c5fc37f07ff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:09.857320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f086233604a49498a6709a26a89b452', 'request_id': 'req-98022223-98f8-46e4-83cf-4c5fc37f07ff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:09.872790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] The token cache is being invalidate because project 8f086233604a49498a6709a26a89b452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:09.873203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:09.874010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-98022223-98f8-46e4-83cf-4c5fc37f07ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:09.875731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1200/2402] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:09 2022] DELETE /identity/v3/projects/8f086233604a49498a6709a26a89b452 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:09.881324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:09.881845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:09.882079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:09.911313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:09.913529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:09.983729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1203/2403] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:09.992173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:09.992595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:09.992972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.016887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:10.018451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:10.031493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1201/2404] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:10.032240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.042671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678044-6d2a-4cbc-8a07-3545316fa152 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.049361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.050350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.051315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.051881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.060092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:22:10.060092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.060092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.073673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1204/2405] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:22:10.077998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.086882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a368ea6-410b-4ce6-90a5-bae3f8cccede None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.093146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.093967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.094445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.094817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.095680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.097731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.113545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:10.190335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1202/2406] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:22:10 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:10.194635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.205954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98473680-0baa-4c54-b81c-c9daaa087f0e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.238249 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:10.265654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.266951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.267244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.267554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.268202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.270289 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.374837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1205/2407] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:22:10 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:10.376634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.394801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ef62-5f8b-4fcd-909e-ef2396d74fb1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.404436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.405152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.405554 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.405914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.406761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.409025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.422352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1203/2408] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:10.426088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.436290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc15086-06e0-4d7e-b88f-51cab391225c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.443413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac812a-ff4b-4c3c-82b3-2d080e139281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.444061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac812a-ff4b-4c3c-82b3-2d080e139281 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.444346 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac812a-ff4b-4c3c-82b3-2d080e139281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.444674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac812a-ff4b-4c3c-82b3-2d080e139281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.445298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac812a-ff4b-4c3c-82b3-2d080e139281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.446976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac812a-ff4b-4c3c-82b3-2d080e139281 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.459496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1206/2409] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:10.463597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.474436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2e52f-3552-4c50-adcc-f9916e357a66 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.479445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a119e4b-13c6-45ea-9d31-5cce0ef21009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.480121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a119e4b-13c6-45ea-9d31-5cce0ef21009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.480405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a119e4b-13c6-45ea-9d31-5cce0ef21009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.480681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a119e4b-13c6-45ea-9d31-5cce0ef21009 admin admin] PATH_INFO: `/v3/projects/0a9a2f77c5d942c88896fa13ccd57615/users/4bf8ab0532084314aca2424960ab88af/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.505196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac13a3-5a85-41e5-9707-b2f78d1ff908 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.505427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a119e4b-13c6-45ea-9d31-5cce0ef21009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a2f77c5d942c88896fa13ccd57615, user_id=4bf8ab0532084314aca2424960ab88af, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.505532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac13a3-5a85-41e5-9707-b2f78d1ff908 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.505532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac13a3-5a85-41e5-9707-b2f78d1ff908 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.508075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1207/2410] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:22:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 10:22:10.508613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a119e4b-13c6-45ea-9d31-5cce0ef21009 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.514567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.514935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.515343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.539114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] MFA Rules not processed for user `5889d3606c904dac9b4674efed1f94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:10.541223 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:10.587828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1204/2411] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:10 2022] PUT /identity/v3/projects/0a9a2f77c5d942c88896fa13ccd57615/users/4bf8ab0532084314aca2424960ab88af/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:10.599027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.599413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.599745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.600231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1208/2412] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:10.621716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.623104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:10.630695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] MFA Rules not processed for user `4bf8ab0532084314aca2424960ab88af`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:10.632010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:10.634175 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d98f1-88dc-47f0-9371-72225c96f5a6 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.696923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2adf7-e3c4-491f-b6b8-951ff040a46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.697462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2adf7-e3c4-491f-b6b8-951ff040a46c service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.698039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2adf7-e3c4-491f-b6b8-951ff040a46c service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.698314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2adf7-e3c4-491f-b6b8-951ff040a46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.703352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1205/2413] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:22:10.707355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2adf7-e3c4-491f-b6b8-951ff040a46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.708966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2adf7-e3c4-491f-b6b8-951ff040a46c service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.713473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.714849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:10.717713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1209/2414] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:22:10.777837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abccd5-80ee-4eb1-9c69-0ea312c9ffdf None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.782734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.783466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.783813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.784218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.794425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.796072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.807204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1206/2415] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:10.824904 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788be853-7bef-4b50-b24b-4e9d162aaa01 None None] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.825281 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788be853-7bef-4b50-b24b-4e9d162aaa01 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.825630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788be853-7bef-4b50-b24b-4e9d162aaa01 None None] PATH_INFO: `/` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.828270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1210/2416] 10.222.0.62 () {54 vars in 921 bytes} [Tue Sep 13 10:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 10:22:10.829911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.833568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.833930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.834278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.859077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] MFA Rules not processed for user `81eb5931848749a1ad22d9d228899cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:10.861071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:10.892453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:10.905737 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec2e5f-5300-4111-8e9d-55d97a34dab5 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.911270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.912035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.912324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.912596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] PATH_INFO: `/v3/users/4bf8ab0532084314aca2424960ab88af` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.913755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf8ab0532084314aca2424960ab88af)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.915949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:10.925324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:10.953322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1211/2417] 10.222.0.62 () {58 vars in 1018 bytes} [Tue Sep 13 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:22:10.963995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:10.973023 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07504bb-8468-4735-a558-63d26ae0005f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:10.976746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb1e29-fa3a-497f-950f-612f2a70c2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:10.977343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb1e29-fa3a-497f-950f-612f2a70c2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:10.977640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb1e29-fa3a-497f-950f-612f2a70c2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:10.977913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb1e29-fa3a-497f-950f-612f2a70c2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:10.988778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb1e29-fa3a-497f-950f-612f2a70c2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:10.990196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb1e29-fa3a-497f-950f-612f2a70c2f6 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:11.001668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1212/2418] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:22:11.137606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf8ab0532084314aca2424960ab88af'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:11.212433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf8ab0532084314aca2424960ab88af'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:11.213342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf8ab0532084314aca2424960ab88af'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:11.214142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf8ab0532084314aca2424960ab88af'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:11.223698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf8ab0532084314aca2424960ab88af'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:11.257069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1207/2419] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:10 2022] DELETE /identity/v3/users/4bf8ab0532084314aca2424960ab88af => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:22:11.261747 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:11.262039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:11.262313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:11.281802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:11.283610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:11.348863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1213/2420] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:11.369291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:11.376123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:11.442731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c41660-575e-4a5d-aff1-7447ace0dceb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:11.446558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70dc82-8ea0-4623-a950-e1617aebcff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:11.447188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70dc82-8ea0-4623-a950-e1617aebcff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:11.447549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70dc82-8ea0-4623-a950-e1617aebcff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:11.447824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70dc82-8ea0-4623-a950-e1617aebcff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:11.457775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70dc82-8ea0-4623-a950-e1617aebcff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:11.459196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70dc82-8ea0-4623-a950-e1617aebcff5 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:11.479439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1208/2421] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:11.938696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:11.941053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:11.953863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d57eec-bd84-480c-a0b3-a18a2e4088e7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:11.966114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:11.972879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:11.973182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:11.973465 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] PATH_INFO: `/v3/projects/0a9a2f77c5d942c88896fa13ccd57615` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:11.975285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9a2f77c5d942c88896fa13ccd57615)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:11.977976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.111063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9a2f77c5d942c88896fa13ccd57615', 'request_id': 'req-f7a18fcc-5e40-402b-b228-c9599a44b60a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:12.118682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9a2f77c5d942c88896fa13ccd57615', 'request_id': 'req-f7a18fcc-5e40-402b-b228-c9599a44b60a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:12.136917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.137401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.137486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.161571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:12.162777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:12.191570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9a2f77c5d942c88896fa13ccd57615', 'request_id': 'req-f7a18fcc-5e40-402b-b228-c9599a44b60a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:12.203228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] The token cache is being invalidate because project 0a9a2f77c5d942c88896fa13ccd57615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:12.203526 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:12.204882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f7a18fcc-5e40-402b-b228-c9599a44b60a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:12.206400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1214/2422] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:11 2022] DELETE /identity/v3/projects/0a9a2f77c5d942c88896fa13ccd57615 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:12.211114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267e6d5-2940-4dae-a8a9-6586050d7f23 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.211480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267e6d5-2940-4dae-a8a9-6586050d7f23 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.211754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267e6d5-2940-4dae-a8a9-6586050d7f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.220311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1209/2423] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:22:12.232317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.232451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e267e6d5-2940-4dae-a8a9-6586050d7f23 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:12.233667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e267e6d5-2940-4dae-a8a9-6586050d7f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:12.233779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:12.256130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:12.294847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1215/2424] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:12.304284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.304574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.305021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.314433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1d50-b507-455f-a2b5-2a9923cbb377 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.319622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.320239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.320521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.320794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.321406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:12.327509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:12.328714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:12.329250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.330837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.336734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1216/2425] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:22:12.339078 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1210/2426] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 10:22:12.341096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.355075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce75dc-c2a8-4b9b-9d1a-253be97185ba None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.355737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.356357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.356624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.356897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.357605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:22:12.357909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.359538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.371815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1217/2427] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:22:12.376815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.387901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f3a7b-d138-4905-8f51-0a516010cc34 service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.393908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.394518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.394793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.395059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.395967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.401049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.521375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1211/2428] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:22:12 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:12.523406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.537649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ada5-6d83-488f-a939-a1cd78ec7b36 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.629075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.629808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.630166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.630443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.631195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.633589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.719992 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1218/2429] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:22:12 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:12.723283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.743773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd16b4f-e05c-4cf2-90b4-0ad6a16d58f1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.759154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.760817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.761141 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.761739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.762649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.764425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.785363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1212/2430] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:22:12.789424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.804123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e4365-5d30-4170-81ac-ac446c067ba4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.810619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f61b2-3131-4e9e-8a80-aa5981405e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.811272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f61b2-3131-4e9e-8a80-aa5981405e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.811561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f61b2-3131-4e9e-8a80-aa5981405e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.811839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f61b2-3131-4e9e-8a80-aa5981405e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.812564 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f61b2-3131-4e9e-8a80-aa5981405e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.814500 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f61b2-3131-4e9e-8a80-aa5981405e17 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:12.827271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1219/2431] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:12.832286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:12.842747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68434e8-e637-4ecb-a8ce-741537895fae admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:12.848010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:12.848691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:12.849066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:12.849340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] PATH_INFO: `/v3/projects/dd7abbcf7a0a4ea992a0100b7c6aefa6/users/069ba4bccf644930979ca84b3418ba83/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:12.888511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7abbcf7a0a4ea992a0100b7c6aefa6, user_id=069ba4bccf644930979ca84b3418ba83, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:12.891223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:13.030253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1213/2432] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:12 2022] PUT /identity/v3/projects/dd7abbcf7a0a4ea992a0100b7c6aefa6/users/069ba4bccf644930979ca84b3418ba83/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:13.042869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:13.043434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:13.043546 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:13.088021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] MFA Rules not processed for user `069ba4bccf644930979ca84b3418ba83`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:13.089655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:13.205979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1220/2433] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:13.219153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:13.220942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:13.305545 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0896-761e-4f99-8e68-1fcb792ab2b1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:13.311212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55699e20-57d5-45f8-8c1c-22d9bd99ff34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:13.312131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55699e20-57d5-45f8-8c1c-22d9bd99ff34 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:13.312490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55699e20-57d5-45f8-8c1c-22d9bd99ff34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:13.312694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55699e20-57d5-45f8-8c1c-22d9bd99ff34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:13.323765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55699e20-57d5-45f8-8c1c-22d9bd99ff34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:13.325394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55699e20-57d5-45f8-8c1c-22d9bd99ff34 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:13.337715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1214/2434] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:13.366917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:13.461240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:13.467710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2e4cb-26ea-479c-b0f1-3658dd568d52 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:13.473521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:13.477207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:13.477207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:13.477207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] PATH_INFO: `/v3/users/069ba4bccf644930979ca84b3418ba83` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:13.477207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069ba4bccf644930979ca84b3418ba83)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:13.477892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:13.708146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069ba4bccf644930979ca84b3418ba83'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:13.780348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069ba4bccf644930979ca84b3418ba83'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:13.781060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069ba4bccf644930979ca84b3418ba83'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:13.782511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069ba4bccf644930979ca84b3418ba83'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:13.790570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069ba4bccf644930979ca84b3418ba83'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:13.809577 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1221/2435] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:13 2022] DELETE /identity/v3/users/069ba4bccf644930979ca84b3418ba83 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:13.814307 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:13.814809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:13.815094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:13.834951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:13.836402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:13.886916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:13.903289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1215/2436] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:13.913766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:13.915292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:13.985768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b8a9-d6ae-4556-8059-11a9d9fc1cb1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:13.989453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ab9bb-3fb9-44ec-8f1b-f0d7931e845a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:13.990061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab9bb-3fb9-44ec-8f1b-f0d7931e845a service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:13.990341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab9bb-3fb9-44ec-8f1b-f0d7931e845a service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:13.990634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab9bb-3fb9-44ec-8f1b-f0d7931e845a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:14.001552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ab9bb-3fb9-44ec-8f1b-f0d7931e845a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:14.003030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ab9bb-3fb9-44ec-8f1b-f0d7931e845a service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:14.013310 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1222/2437] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:14.497350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:14.499128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:14.514558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9a22e-3c15-425d-88be-0bc79294ece0 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:14.520196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:14.521163 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:14.521834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:14.522326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] PATH_INFO: `/v3/projects/dd7abbcf7a0a4ea992a0100b7c6aefa6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:14.523777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7abbcf7a0a4ea992a0100b7c6aefa6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:14.526295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:14.632705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7abbcf7a0a4ea992a0100b7c6aefa6', 'request_id': 'req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:14.641790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7abbcf7a0a4ea992a0100b7c6aefa6', 'request_id': 'req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:14.717126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7abbcf7a0a4ea992a0100b7c6aefa6', 'request_id': 'req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:14.729967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] The token cache is being invalidate because project dd7abbcf7a0a4ea992a0100b7c6aefa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:14.730275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:14.730943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c18e4dd6-57c0-4d70-ab54-b9b115caf8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:14.732552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1216/2438] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:14 2022] DELETE /identity/v3/projects/dd7abbcf7a0a4ea992a0100b7c6aefa6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:14.741428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:14.743569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:14.743569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:14.766807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:14.768083 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:14.861410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1223/2439] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:14.869161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:14.869451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:14.869735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:14.897717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:14.898901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:14.911229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1217/2440] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:14.912878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:14.924150 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74d2e8-8f36-4383-9612-a4ab424a2ce3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:14.931275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:14.932047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:14.932359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:14.932660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:14.933353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:22:14.933646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:14.935323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:14.949069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1224/2441] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:22:14.953625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:14.971616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb17c2a-0004-408c-97fa-491c3a9a2c7b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:14.971616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:14.971616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:14.971616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:14.971983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:14.972157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:14.977652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:15.057985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1218/2442] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:22:14 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:22:15.062324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:15.071826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:15.077170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31456b8-8760-4d8c-a250-5f9d24675ee5 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:15.147286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:15.148075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:15.148321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:15.148606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:15.149261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:15.151293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:15.277621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1225/2443] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:22:15 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:22:15.280314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:15.291913 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449adccc-5889-4fd2-9c1e-835692e608e4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:15.296715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:15.297443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:15.297671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:15.297948 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:15.298844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:15.300500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:15.317022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1219/2444] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:22:15.318264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:15.330033 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489e0c6-80cf-4918-80cf-5cc9b86fb08c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:15.336930 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191cc61-8805-4555-8ecf-ef3adc731e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:15.337728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191cc61-8805-4555-8ecf-ef3adc731e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:15.338072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191cc61-8805-4555-8ecf-ef3adc731e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:15.338416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191cc61-8805-4555-8ecf-ef3adc731e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:15.339132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191cc61-8805-4555-8ecf-ef3adc731e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:15.341017 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191cc61-8805-4555-8ecf-ef3adc731e00 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:15.354303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1226/2445] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:15.358661 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:15.371583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd914-55fe-4a2f-af3c-63620a209789 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:15.376677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:15.377512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:15.377895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:15.378265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] PATH_INFO: `/v3/projects/ec6a99b023f744d186ae4603ccf1a1ae/users/8a9df73d59d247d787967bfdb0f8c2b6/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:15.397056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6a99b023f744d186ae4603ccf1a1ae, user_id=8a9df73d59d247d787967bfdb0f8c2b6, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:15.400057 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:15.473736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1220/2446] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:15 2022] PUT /identity/v3/projects/ec6a99b023f744d186ae4603ccf1a1ae/users/8a9df73d59d247d787967bfdb0f8c2b6/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:15.544762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6486-58d4-4610-be91-440df72e2acc None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:15.545053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6486-58d4-4610-be91-440df72e2acc None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:15.545425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6486-58d4-4610-be91-440df72e2acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:15.574183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-53af6486-58d4-4610-be91-440df72e2acc None None] MFA Rules not processed for user `8a9df73d59d247d787967bfdb0f8c2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:15.575486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-53af6486-58d4-4610-be91-440df72e2acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:15.654635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1227/2447] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:15.664092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:15.666032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:15.757140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a6ee5-7554-454b-86a0-c679a84acd4d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:15.761233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177d036-9323-4961-af9a-20ee3d03bf56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:15.761870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177d036-9323-4961-af9a-20ee3d03bf56 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:15.762149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177d036-9323-4961-af9a-20ee3d03bf56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:15.762426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177d036-9323-4961-af9a-20ee3d03bf56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:15.772559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177d036-9323-4961-af9a-20ee3d03bf56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:15.775983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177d036-9323-4961-af9a-20ee3d03bf56 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:15.791868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1221/2448] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:25.647570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:25.783282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:27.736327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6486-58d4-4610-be91-440df72e2acc None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:27.800344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6486-58d4-4610-be91-440df72e2acc None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:27.805067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:27.805739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:27.806014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:27.806327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] PATH_INFO: `/v3/users/8a9df73d59d247d787967bfdb0f8c2b6` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:27.807487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9df73d59d247d787967bfdb0f8c2b6)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:27.812263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:28.012308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9df73d59d247d787967bfdb0f8c2b6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:28.088026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9df73d59d247d787967bfdb0f8c2b6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:28.088654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9df73d59d247d787967bfdb0f8c2b6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:28.089261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9df73d59d247d787967bfdb0f8c2b6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:28.097585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9df73d59d247d787967bfdb0f8c2b6'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:28.114780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1228/2449] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:27 2022] DELETE /identity/v3/users/8a9df73d59d247d787967bfdb0f8c2b6 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:28.119423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:28.119895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:28.119990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:28.139520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:28.141383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:28.200164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1222/2450] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:22:28.210594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:28.211967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:28.284699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932583f-e4a7-43c6-a720-37073a8aaf3c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:28.289208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77ffbf-ebce-4cca-9756-2be0b6fa2f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:28.289817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77ffbf-ebce-4cca-9756-2be0b6fa2f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:28.290097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77ffbf-ebce-4cca-9756-2be0b6fa2f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:28.290370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77ffbf-ebce-4cca-9756-2be0b6fa2f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:28.299512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77ffbf-ebce-4cca-9756-2be0b6fa2f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:28.300941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77ffbf-ebce-4cca-9756-2be0b6fa2f36 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:28.318486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1229/2451] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:22:28.721022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:28.722457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:28.737568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181f46e-226c-49a6-aa4b-91c81983a30f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:28.744337 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:28.744853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:28.745558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:28.745558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] PATH_INFO: `/v3/projects/ec6a99b023f744d186ae4603ccf1a1ae` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:28.746949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6a99b023f744d186ae4603ccf1a1ae)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:28.751944 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:28.916453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6a99b023f744d186ae4603ccf1a1ae', 'request_id': 'req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:28.923500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6a99b023f744d186ae4603ccf1a1ae', 'request_id': 'req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:29.010835 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6a99b023f744d186ae4603ccf1a1ae', 'request_id': 'req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:29.021638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] The token cache is being invalidate because project ec6a99b023f744d186ae4603ccf1a1ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:29.021922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:29.022534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8353b9e6-cb7a-4166-8ab1-034c676a8d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:29.024056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1223/2452] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:28 2022] DELETE /identity/v3/projects/ec6a99b023f744d186ae4603ccf1a1ae => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:29.029190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.029643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.029735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.049581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:29.051298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:29.067603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:29.117263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1230/2453] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:29.119059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.119372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.119646 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.148128 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:29.149418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:29.163586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1224/2454] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:29.164362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.175663 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554433a-5b4a-4a4d-abed-00cc0832559c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:29.180442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:29.181209 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.181597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.181893 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.182615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:22:29.182986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:29.184726 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:29.198437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1231/2455] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:22:29.207704 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.218303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06b16-9b14-45bc-a53b-7d281a6c67ba None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:29.224246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:29.225183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.225586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.225967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.232537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:29.232537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:29.317674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1225/2456] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:22:29 2022] POST /identity/v3/projects => generated 395 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:29.322318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.336486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d8fda-baf9-4654-a97e-083406aa16da admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:29.487707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:29.487707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.487707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.487707 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.488700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:29.490799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:29.603892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1232/2457] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:22:29 2022] POST /identity/v3/users => generated 298 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:22:29.608214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.618983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638008d-9f05-43a5-8d82-d83e4ff68a5b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:29.625779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:29.626496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.626784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.627060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.627801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:29.629537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:29.643431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1226/2458] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:29.644883 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.655623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5bfc0-8e85-4f6b-a0fb-27bc13bc07a7 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:29.660985 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af1b5f-d5ff-44bc-b7ff-5fa290571b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:29.661590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1b5f-d5ff-44bc-b7ff-5fa290571b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.661871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1b5f-d5ff-44bc-b7ff-5fa290571b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.662144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1b5f-d5ff-44bc-b7ff-5fa290571b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.662784 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af1b5f-d5ff-44bc-b7ff-5fa290571b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:29.664418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af1b5f-d5ff-44bc-b7ff-5fa290571b72 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:29.675371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1233/2459] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:29.679607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.689665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554f83b-1b41-4d55-8ad5-ef1b66d68dcc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:29.694420 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:29.695066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.695379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.695654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] PATH_INFO: `/v3/projects/ccc72db0cd7948dfbf321435d8de1d82/users/5c1aaec490a0476e816239f44c14e114/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.712588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc72db0cd7948dfbf321435d8de1d82, user_id=5c1aaec490a0476e816239f44c14e114, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:29.715419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:29.836794 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1227/2460] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:29 2022] PUT /identity/v3/projects/ccc72db0cd7948dfbf321435d8de1d82/users/5c1aaec490a0476e816239f44c14e114/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:29.849168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf484-17ef-4f14-9aba-d868e54e4ba1 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:29.849522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf484-17ef-4f14-9aba-d868e54e4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:29.849868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf484-17ef-4f14-9aba-d868e54e4ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:29.881425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-66baf484-17ef-4f14-9aba-d868e54e4ba1 None None] MFA Rules not processed for user `5c1aaec490a0476e816239f44c14e114`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:29.882644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-66baf484-17ef-4f14-9aba-d868e54e4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:29.950171 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1234/2461] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:29.962121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:29.963918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:29.986968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:30.023217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912e5dd-1793-43aa-b62d-57b0e52d59c3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:30.027330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:30.027942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:30.028226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:30.028496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:30.036977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:30.038419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:30.047805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1228/2462] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:31.495923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:31.496616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:31.496616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:31.518156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:31.519435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:31.533323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:31.577151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1235/2463] 10.222.0.62 () {58 vars in 967 bytes} [Tue Sep 13 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:31.590488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:31.591998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:31.662776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7817c-ac97-408a-bec5-b18db3f6329c service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:31.667820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d04d7-7289-4f46-b72b-f300b28da190 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:31.668426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d04d7-7289-4f46-b72b-f300b28da190 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:31.668709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d04d7-7289-4f46-b72b-f300b28da190 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:31.668984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d04d7-7289-4f46-b72b-f300b28da190 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:31.677810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d04d7-7289-4f46-b72b-f300b28da190 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:31.678750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d04d7-7289-4f46-b72b-f300b28da190 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:31.686701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1229/2464] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:22:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:22:39.207870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:39.278272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5140f-013d-4774-8678-500d2b86bcbe None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:39.283073 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:39.286522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:39.286522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:39.286522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] PATH_INFO: `/v3/users/5c1aaec490a0476e816239f44c14e114` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:39.286522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1aaec490a0476e816239f44c14e114)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:39.287643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:39.476752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1aaec490a0476e816239f44c14e114'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:39.533657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1aaec490a0476e816239f44c14e114'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:39.534285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1aaec490a0476e816239f44c14e114'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:39.534888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1aaec490a0476e816239f44c14e114'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:39.542168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1aaec490a0476e816239f44c14e114'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:39.560155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1236/2465] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:39 2022] DELETE /identity/v3/users/5c1aaec490a0476e816239f44c14e114 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:39.565352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:39.565855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:39.565961 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:39.584692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:39.586017 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:39.644755 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1230/2466] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:39.659502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:39.659502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:39.702889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:39.714268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aeba6-888d-4968-92f5-9199176d4b13 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:39.717952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:39.718565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:39.718844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:39.719122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:39.728743 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:39.730443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:39.743687 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1237/2467] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:40.149747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:40.151550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:40.165595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f3309-9cd3-409b-bcdb-2e2f9955d293 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:40.170500 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:40.171272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:40.171622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:40.172081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] PATH_INFO: `/v3/projects/ccc72db0cd7948dfbf321435d8de1d82` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:40.173358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc72db0cd7948dfbf321435d8de1d82)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:40.175795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:40.253264 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:40.254519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc72db0cd7948dfbf321435d8de1d82', 'request_id': 'req-fd038321-e7d2-4047-bf0f-d40d4a629731'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:40.260599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc72db0cd7948dfbf321435d8de1d82', 'request_id': 'req-fd038321-e7d2-4047-bf0f-d40d4a629731'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:40.345377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc72db0cd7948dfbf321435d8de1d82', 'request_id': 'req-fd038321-e7d2-4047-bf0f-d40d4a629731'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:40.356150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] The token cache is being invalidate because project ccc72db0cd7948dfbf321435d8de1d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:40.356444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:40.357130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:40.358612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1231/2468] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:40 2022] DELETE /identity/v3/projects/ccc72db0cd7948dfbf321435d8de1d82 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:22:49.813947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=7 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:50.370379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:51.578392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:51.587371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954b300-9e99-49c1-865b-a2c5595d54d4 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:51.637386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:51.637993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:51.638280 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:51.638554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:51.640348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:51.702008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:51.703502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:51.713236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1238/2469] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:54.697369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:54.699007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:54.778091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd038321-e7d2-4047-bf0f-d40d4a629731 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:54.784224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:54.784834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:54.785118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:54.785395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:54.787145 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:54.874770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:54.878259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:54.892361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1232/2470] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:55.451778 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:55.453263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:55.533144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855936b2-ab48-48fb-ae4e-918d576166a0 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:55.538330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:55.538967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:55.539278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:55.539568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:55.541306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:55.560610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:55.563745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:55.577100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1239/2471] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:55.582797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:55.584583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:55.608214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb04ff-9acd-4ad3-a7e0-b45bf75ca20e service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:55.613095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:55.614137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:55.614427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:55.614705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] PATH_INFO: `/v3/users/4067d48268f84854af4d672ab3f9dc65` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:55.616300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4067d48268f84854af4d672ab3f9dc65)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:55.620187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:55.818758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4067d48268f84854af4d672ab3f9dc65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:55.910596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4067d48268f84854af4d672ab3f9dc65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:55.911247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4067d48268f84854af4d672ab3f9dc65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:55.911867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4067d48268f84854af4d672ab3f9dc65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:55.919601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d70ba10-be06-4e9b-86f1-f4fcdaa810fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4067d48268f84854af4d672ab3f9dc65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:55.937342 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1233/2472] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:22:55 2022] DELETE /identity/v3/users/4067d48268f84854af4d672ab3f9dc65 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:22:56.197823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:56.209394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4d03-ed9e-4fb4-96a2-3b5aa81bc0b7 service placement] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:56.250329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:56.268153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:56.268961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:56.269081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:56.269329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] PATH_INFO: `/v3/projects/66658bb98ea64e319ed948b8f279ed5f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:56.270448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=66658bb98ea64e319ed948b8f279ed5f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:56.272369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:56.470181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66658bb98ea64e319ed948b8f279ed5f', 'request_id': 'req-0f9285e5-f054-4a86-bef1-d86d12e5e22b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:56.476476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66658bb98ea64e319ed948b8f279ed5f', 'request_id': 'req-0f9285e5-f054-4a86-bef1-d86d12e5e22b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:56.552278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66658bb98ea64e319ed948b8f279ed5f', 'request_id': 'req-0f9285e5-f054-4a86-bef1-d86d12e5e22b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:56.563126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] The token cache is being invalidate because project 66658bb98ea64e319ed948b8f279ed5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:22:56.563536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:56.564159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0f9285e5-f054-4a86-bef1-d86d12e5e22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:22:56.565678 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1240/2473] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:22:56 2022] DELETE /identity/v3/projects/66658bb98ea64e319ed948b8f279ed5f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:56.570529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:56.570819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:56.571093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:56.590808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:56.592048 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:56.596799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:56.651031 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1234/2474] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:56.659911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:56.660205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:56.660490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:56.680322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:56.681569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:56.689627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1241/2475] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:56.693772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:56.704491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94da6d-c59a-4299-9585-ee40cd6c9a2f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:56.709214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:56.709837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:56.710123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:56.710399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:56.711046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:22:56.711442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:56.713123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:56.728274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1235/2476] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:22:56.730118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:56.739744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade7efe-6a20-4c1e-b75e-0a70f329e727 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:56.744790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:56.745425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:56.745730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:56.746041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:56.746659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:56.749009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:56.859256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1242/2477] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:22:56 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:56.861130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:56.871042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c057f0-fd08-4daf-983d-4b52e941566a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:56.929385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:56.929994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:56.930277 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:56.930553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:56.931235 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:56.933161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.027796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1236/2478] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:22:56 2022] POST /identity/v3/users => generated 282 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:57.031901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.042225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7c390-aabb-45ad-960b-de5ef420ce39 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.051324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.051324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.051324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.051324 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.051727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.051727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.065649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1243/2479] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:22:57.066834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.077205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab44cb-44eb-43b1-a1a0-6069600a1618 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.082683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae82955-6cce-4ba3-ae45-a2f57d8f3e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.083352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82955-6cce-4ba3-ae45-a2f57d8f3e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.083720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82955-6cce-4ba3-ae45-a2f57d8f3e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.084206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82955-6cce-4ba3-ae45-a2f57d8f3e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.084963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae82955-6cce-4ba3-ae45-a2f57d8f3e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.086594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae82955-6cce-4ba3-ae45-a2f57d8f3e15 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.097868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1237/2480] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:57.102259 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.112682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc3c76-2d15-49d3-95d0-1f5306028930 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.117318 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.117965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.118248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.118522 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] PATH_INFO: `/v3/projects/380de8a28361409e8822a7ee63c103d9/users/58efaad6e90f4c7f96d49ecee9e53b1d/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.134497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=380de8a28361409e8822a7ee63c103d9, user_id=58efaad6e90f4c7f96d49ecee9e53b1d, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.137393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.230897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1244/2481] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:57 2022] PUT /identity/v3/projects/380de8a28361409e8822a7ee63c103d9/users/58efaad6e90f4c7f96d49ecee9e53b1d/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:57.241698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.241990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.242255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.271068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] MFA Rules not processed for user `58efaad6e90f4c7f96d49ecee9e53b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:57.272375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:57.338700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1238/2482] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:57.343402 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.407989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168affd-320f-4bef-9500-a8c9107d7aac admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.412634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.413237 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.413531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.413789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.414416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.416365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.427669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:57.502055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1245/2483] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:22:57 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:22:57.506102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.512805 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:57.517134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7a408-d167-4f99-bb5e-7029cb154aec None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.579039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.579758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.580050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.580471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.581460 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.583526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.664952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1239/2484] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:22:57 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:22:57.669070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.679951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c887-4abb-4c77-8995-ad55f60470ea admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.684542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.685178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.687087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.687087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.687087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.688421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.698229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1246/2485] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:57.702521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.712593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de91a24-b1e7-4c08-877a-641300c7bdea admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.717227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.717875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.718159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.718436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] PATH_INFO: `/v3/projects/c62853774e8b4ad1832d1fcff4f5fc53/users/e6fbd79528f04ffba0307121d494372d/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.736456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62853774e8b4ad1832d1fcff4f5fc53, user_id=e6fbd79528f04ffba0307121d494372d, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.739115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.841267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1240/2486] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:57 2022] PUT /identity/v3/projects/c62853774e8b4ad1832d1fcff4f5fc53/users/e6fbd79528f04ffba0307121d494372d/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:57.845925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.856112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba687-3a22-4f5f-b0e5-7e14116354c8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.913812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ab99c-a4ac-40aa-b98b-3a41ba0a84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.914407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ab99c-a4ac-40aa-b98b-3a41ba0a84d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.914655 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ab99c-a4ac-40aa-b98b-3a41ba0a84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.914934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ab99c-a4ac-40aa-b98b-3a41ba0a84d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.915621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ab99c-a4ac-40aa-b98b-3a41ba0a84d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.917228 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ab99c-a4ac-40aa-b98b-3a41ba0a84d1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.927518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1247/2487] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:22:57.931644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:57.941987 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d8485-5d1b-4bfd-a40a-2f438175705b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:57.946823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:57.947533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.947870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.948151 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] PATH_INFO: `/v3/projects/c62853774e8b4ad1832d1fcff4f5fc53/users/e6fbd79528f04ffba0307121d494372d/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:57.950089 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62853774e8b4ad1832d1fcff4f5fc53, user_id=e6fbd79528f04ffba0307121d494372d, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:57.952808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:57.963654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1241/2488] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:22:57 2022] PUT /identity/v3/projects/c62853774e8b4ad1832d1fcff4f5fc53/users/e6fbd79528f04ffba0307121d494372d/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:22:57.975767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:57.976061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:57.977444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:58.003772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] MFA Rules not processed for user `e6fbd79528f04ffba0307121d494372d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:22:58.004995 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:58.074216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1248/2489] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:22:58.085992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:58.087720 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:58.143958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3373b-f26e-4eee-b894-b5422f2e81ac admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:58.147666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:58.148283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:58.148548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:58.148819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:58.158658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:58.160107 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:58.169579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1242/2490] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:22:58.366739 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:22:58.368217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:22:58.380860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9ffee-e9d9-49a4-a201-aa8f489a4e7e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:22:58.384579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82becee-aa3f-497a-b37d-16307b6c3a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:22:58.385185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82becee-aa3f-497a-b37d-16307b6c3a69 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:22:58.389955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82becee-aa3f-497a-b37d-16307b6c3a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:22:58.389955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82becee-aa3f-497a-b37d-16307b6c3a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:22:58.441608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82becee-aa3f-497a-b37d-16307b6c3a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:22:58.442956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82becee-aa3f-497a-b37d-16307b6c3a69 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:22:58.447440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:22:58.451957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1249/2491] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:22:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:08.165326 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:10.281201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:10.343240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa32f7-7667-4499-9453-2a96b4c30e00 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:10.347955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:10.348601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:10.348888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:10.349165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] PATH_INFO: `/v3/users/58efaad6e90f4c7f96d49ecee9e53b1d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:10.350320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58efaad6e90f4c7f96d49ecee9e53b1d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:10.352274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:10.543764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58efaad6e90f4c7f96d49ecee9e53b1d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:10.637989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58efaad6e90f4c7f96d49ecee9e53b1d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:10.638657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58efaad6e90f4c7f96d49ecee9e53b1d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:10.639297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58efaad6e90f4c7f96d49ecee9e53b1d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:10.647773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58efaad6e90f4c7f96d49ecee9e53b1d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:10.666535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1243/2492] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:10 2022] DELETE /identity/v3/users/58efaad6e90f4c7f96d49ecee9e53b1d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:10.671401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:10.671874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:10.672353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:10.693115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:10.694354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:10.754793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1250/2493] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:10.764985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:10.766507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:10.824047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3846b7-ac14-482a-9afe-c42bf44c9273 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:10.827633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:10.828242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:10.828525 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:10.828797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:10.837931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:10.839382 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:10.848746 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1244/2494] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:11.279605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:11.281134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:11.293971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f8fcc-41d1-4a4c-82cc-edcc1d0fe341 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:11.299309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:11.299859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:11.300144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:11.300421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] PATH_INFO: `/v3/projects/380de8a28361409e8822a7ee63c103d9` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:11.301554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380de8a28361409e8822a7ee63c103d9)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:11.303543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:11.436954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380de8a28361409e8822a7ee63c103d9', 'request_id': 'req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.443294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380de8a28361409e8822a7ee63c103d9', 'request_id': 'req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.544000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380de8a28361409e8822a7ee63c103d9', 'request_id': 'req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.554672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] The token cache is being invalidate because project 380de8a28361409e8822a7ee63c103d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:11.554957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.555592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.557091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1251/2495] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:11 2022] DELETE /identity/v3/projects/380de8a28361409e8822a7ee63c103d9 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:11.561092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:11.570586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d1ac1-093f-4454-9a27-585e9868db92 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:11.588675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:11.627550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:11.628364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:11.628641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:11.628918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] PATH_INFO: `/v3/users/e6fbd79528f04ffba0307121d494372d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:11.630074 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fbd79528f04ffba0307121d494372d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:11.632027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:11.799402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fbd79528f04ffba0307121d494372d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.866006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fbd79528f04ffba0307121d494372d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.866638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fbd79528f04ffba0307121d494372d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.867276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fbd79528f04ffba0307121d494372d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.874479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-59177f7e-8b94-48d6-aa28-e56dccf1e7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fbd79528f04ffba0307121d494372d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:11.892930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1245/2496] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:11 2022] DELETE /identity/v3/users/e6fbd79528f04ffba0307121d494372d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:12.302489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:12.303929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:12.317910 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50e5f4-6860-4f9b-8c98-76d3578d8d73 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:12.373744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:12.374424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:12.374711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:12.374994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] PATH_INFO: `/v3/projects/c62853774e8b4ad1832d1fcff4f5fc53` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:12.376835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62853774e8b4ad1832d1fcff4f5fc53)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:12.378783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:12.465505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:12.481445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62853774e8b4ad1832d1fcff4f5fc53', 'request_id': 'req-2c540419-5440-40eb-99de-0d6591f70c6d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:12.488330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62853774e8b4ad1832d1fcff4f5fc53', 'request_id': 'req-2c540419-5440-40eb-99de-0d6591f70c6d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:12.555136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62853774e8b4ad1832d1fcff4f5fc53', 'request_id': 'req-2c540419-5440-40eb-99de-0d6591f70c6d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:12.567275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] The token cache is being invalidate because project c62853774e8b4ad1832d1fcff4f5fc53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:12.567427 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:12.568043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2c540419-5440-40eb-99de-0d6591f70c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:12.569538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1252/2497] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:12 2022] DELETE /identity/v3/projects/c62853774e8b4ad1832d1fcff4f5fc53 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:12.574508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:12.574964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:12.578904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:12.595822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:12.597096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:12.661976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1246/2498] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:12.664562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:12.664858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:12.665138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:12.686181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:12.687435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:12.707364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1253/2499] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:12.707478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:12.711010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992adf4e-cb79-4c5f-a2fa-265d346ed254 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:12.715923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:12.716526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:12.716809 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:12.717081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:12.717724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:12.718028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:12.720262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:12.735642 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1247/2500] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:12.738993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:12.748777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1854b-fa4c-4aea-a94c-64e0f6fe55b7 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:12.753543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:12.754140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:12.754419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:12.754684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:12.755344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:12.757240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:12.840200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1254/2501] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:12 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:12.845859 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:12.856122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d35b60-762b-4f30-a04b-96742b068a3c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:12.885334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:12.917068 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:12.917675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:12.918083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:12.918387 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:12.919033 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:12.921018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:12.992679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1248/2502] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:12 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:12.996807 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.007149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c116b5b-57ef-4846-99c7-8845698ff67e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.011738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.012327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.012605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.012876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.013512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.015120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.025779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1255/2503] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:13.030060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.040687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52446594-d9e7-49ef-a4ba-e991f5c21f1e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.045758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877a8f0-0b17-4f12-954f-693dd030f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.046362 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877a8f0-0b17-4f12-954f-693dd030f265 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.046647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877a8f0-0b17-4f12-954f-693dd030f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.046923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877a8f0-0b17-4f12-954f-693dd030f265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.047606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877a8f0-0b17-4f12-954f-693dd030f265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.049196 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877a8f0-0b17-4f12-954f-693dd030f265 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.060171 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1249/2504] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:13.064431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.075015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22369-dd4c-4d25-8f66-ac31219f7226 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.080406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.081018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.081298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.081580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] PATH_INFO: `/v3/projects/c9d7bc6bdc2a45b38c1794d5bb0f9669/users/4011b03b1cf04162a76f5b1e5ca787bb/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.097162 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d7bc6bdc2a45b38c1794d5bb0f9669, user_id=4011b03b1cf04162a76f5b1e5ca787bb, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.099853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.208178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1256/2505] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:13 2022] PUT /identity/v3/projects/c9d7bc6bdc2a45b38c1794d5bb0f9669/users/4011b03b1cf04162a76f5b1e5ca787bb/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:13.220320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.220616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.220886 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.250504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] MFA Rules not processed for user `4011b03b1cf04162a76f5b1e5ca787bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:13.251783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:13.319195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1250/2506] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:13.323575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.384820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aac025-03a7-417e-a511-c220e92b0867 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.394147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.394754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.395042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.395368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.396019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.397947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.481702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1257/2507] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:13 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:13.485758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.496741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea37e4-9271-4352-9aaf-64e314675e2c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.552951 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.553557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.553843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.554121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.554750 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.556736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.669681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1251/2508] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:13 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:13.674090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.684683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985254a-61c6-4f2d-8fbc-91eedbbcd6de admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.690030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.690631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.690914 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.691216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.691858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.693490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.704117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1258/2509] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:13.708006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.718158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29a3f-e204-4e04-b44c-36cf37f5efa3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.723030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.723732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.724019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.724297 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] PATH_INFO: `/v3/projects/dd7f63169add4468a9b4b2539063c78f/users/9a8e6e177da0483ba15cf4be75becc82/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.739745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7f63169add4468a9b4b2539063c78f, user_id=9a8e6e177da0483ba15cf4be75becc82, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.742448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.806715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1252/2510] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:13 2022] PUT /identity/v3/projects/dd7f63169add4468a9b4b2539063c78f/users/9a8e6e177da0483ba15cf4be75becc82/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:13.810651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.821580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2b61b-7e85-4c32-a529-da5cd8f6f171 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.841468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:13.879478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f6bcf-f35a-490d-9ab1-157a0444a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.880081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f6bcf-f35a-490d-9ab1-157a0444a2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.880366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f6bcf-f35a-490d-9ab1-157a0444a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.880641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f6bcf-f35a-490d-9ab1-157a0444a2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.881279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f6bcf-f35a-490d-9ab1-157a0444a2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.882912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f6bcf-f35a-490d-9ab1-157a0444a2f3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.892614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1259/2511] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:13.896742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:13.903375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:13.908708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73783-a8fd-4308-a9ba-b12200366858 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:13.913440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:13.914085 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.915654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.915654 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] PATH_INFO: `/v3/projects/dd7f63169add4468a9b4b2539063c78f/users/9a8e6e177da0483ba15cf4be75becc82/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.916617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7f63169add4468a9b4b2539063c78f, user_id=9a8e6e177da0483ba15cf4be75becc82, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:13.919262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:13.929764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1253/2512] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:13 2022] PUT /identity/v3/projects/dd7f63169add4468a9b4b2539063c78f/users/9a8e6e177da0483ba15cf4be75becc82/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:13.940377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:13.945115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:13.945115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:13.971449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] MFA Rules not processed for user `9a8e6e177da0483ba15cf4be75becc82`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:13.972719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:14.043625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1260/2513] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:14.055352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:14.057044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:14.115854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230654d-c3f1-4927-a3e8-8169a49393b0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:14.119550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229c7de-6fa8-4125-8008-c83b4d4c1df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:14.120155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229c7de-6fa8-4125-8008-c83b4d4c1df8 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:14.120438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229c7de-6fa8-4125-8008-c83b4d4c1df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:14.120710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229c7de-6fa8-4125-8008-c83b4d4c1df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:14.129550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229c7de-6fa8-4125-8008-c83b4d4c1df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:14.130976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229c7de-6fa8-4125-8008-c83b4d4c1df8 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:14.140315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1254/2514] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:14.169146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:14.230806 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89d8d4-e3e5-4385-b3b1-77d7d3be594a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:14.236144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:14.236775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:14.237056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:14.237328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] PATH_INFO: `/v3/users/4011b03b1cf04162a76f5b1e5ca787bb` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:14.238477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4011b03b1cf04162a76f5b1e5ca787bb)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:14.240418 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:14.433916 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4011b03b1cf04162a76f5b1e5ca787bb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:14.435668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:14.492746 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4011b03b1cf04162a76f5b1e5ca787bb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:14.493511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4011b03b1cf04162a76f5b1e5ca787bb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:14.496403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4011b03b1cf04162a76f5b1e5ca787bb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:14.502062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4011b03b1cf04162a76f5b1e5ca787bb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:14.519965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1261/2515] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:14 2022] DELETE /identity/v3/users/4011b03b1cf04162a76f5b1e5ca787bb => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:14.524450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:14.524741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:14.525019 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:14.544581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:14.545796 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:14.603598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1255/2516] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:14.613299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:14.614735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:14.674630 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab76c94-3446-41bd-92e3-7be175db2c22 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:14.678403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:14.679008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:14.679323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:14.683558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:14.688805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:14.690255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:14.701388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1262/2517] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:15.196144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:15.197821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:15.214302 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede1703-7298-4b95-984e-20dadd6830ef None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:15.227943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:15.227943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:15.227943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:15.227943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] PATH_INFO: `/v3/projects/c9d7bc6bdc2a45b38c1794d5bb0f9669` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:15.228645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d7bc6bdc2a45b38c1794d5bb0f9669)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:15.228645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:15.335576 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d7bc6bdc2a45b38c1794d5bb0f9669', 'request_id': 'req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.341657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d7bc6bdc2a45b38c1794d5bb0f9669', 'request_id': 'req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.466721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d7bc6bdc2a45b38c1794d5bb0f9669', 'request_id': 'req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.479178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] The token cache is being invalidate because project c9d7bc6bdc2a45b38c1794d5bb0f9669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:15.482673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.482673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.482673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1256/2518] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:15 2022] DELETE /identity/v3/projects/c9d7bc6bdc2a45b38c1794d5bb0f9669 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:15.486186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:15.498009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570484a-2de1-4df3-a8a8-e98a47430720 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:15.578843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:15.579660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:15.579943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:15.580312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] PATH_INFO: `/v3/users/9a8e6e177da0483ba15cf4be75becc82` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:15.581344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8e6e177da0483ba15cf4be75becc82)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:15.583304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:15.747333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8e6e177da0483ba15cf4be75becc82'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.820986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8e6e177da0483ba15cf4be75becc82'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.821661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8e6e177da0483ba15cf4be75becc82'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.822293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8e6e177da0483ba15cf4be75becc82'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.831696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e5377616-c6f4-4578-9dcb-fb915d4fa792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8e6e177da0483ba15cf4be75becc82'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:15.855006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:15.873063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1263/2519] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:15 2022] DELETE /identity/v3/users/9a8e6e177da0483ba15cf4be75becc82 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:16.362941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:16.364552 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:16.394365 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b0306-49d8-4f3c-85f1-bab8ec59a05e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:16.410916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:16.459627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:16.460391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:16.460777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:16.461114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] PATH_INFO: `/v3/projects/dd7f63169add4468a9b4b2539063c78f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:16.462486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7f63169add4468a9b4b2539063c78f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:16.464601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:16.571752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7f63169add4468a9b4b2539063c78f', 'request_id': 'req-17787098-e260-4cef-b249-56fe45b78f55'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:16.580317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7f63169add4468a9b4b2539063c78f', 'request_id': 'req-17787098-e260-4cef-b249-56fe45b78f55'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:16.675752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7f63169add4468a9b4b2539063c78f', 'request_id': 'req-17787098-e260-4cef-b249-56fe45b78f55'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:16.690915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] The token cache is being invalidate because project dd7f63169add4468a9b4b2539063c78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:16.690915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:16.690915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-17787098-e260-4cef-b249-56fe45b78f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:16.695062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1257/2520] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:16 2022] DELETE /identity/v3/projects/dd7f63169add4468a9b4b2539063c78f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:16.697014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:16.697473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:16.697585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:16.733464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:16.733464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:16.811555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1264/2521] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:16.820161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:16.820457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:16.820736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:16.847064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:16.848888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:16.860236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1258/2522] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:16.864643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:16.876434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77fbbe-b68a-4493-be5b-d8d30efbf32b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:16.882093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:16.882804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:16.887983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:16.887983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:16.887983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:16.887983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:16.887983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:16.899360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1265/2523] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:16.903450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:16.919589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff1226-186d-4aa4-8fbe-b8afad4d902e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:16.923331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:16.923972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:16.924309 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:16.924501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:16.925156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:16.927267 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.015275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1259/2524] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:16 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:17.019584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.031706 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c598a-9666-4be3-927b-241c9243453b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.102439 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.103093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.103413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.103718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.104363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.106335 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.180463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1266/2525] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:17 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:17.184583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.195115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d52675-25c2-40be-9a63-5b3cb50ddda2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.200519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.201131 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.201412 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.201683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.202324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.204073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.214229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1260/2526] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:17.218272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.229557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b15e9-38be-4470-8158-b5453def4177 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.235396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9f01d-23e1-4726-bbdb-e35f3615e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.236015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9f01d-23e1-4726-bbdb-e35f3615e385 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.236375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9f01d-23e1-4726-bbdb-e35f3615e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.236600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9f01d-23e1-4726-bbdb-e35f3615e385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.237284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9f01d-23e1-4726-bbdb-e35f3615e385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.238840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9f01d-23e1-4726-bbdb-e35f3615e385 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.253658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1267/2527] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:17.254989 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.266603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b8e45-e2ed-4073-9b50-75f1b23c894e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.271581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.272242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.272523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.272799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] PATH_INFO: `/v3/projects/cb91fdd0ec954db297f44ad5690c7c65/users/a1265b4b6c9f4f3fac7a85fc61a97cf0/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.291026 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb91fdd0ec954db297f44ad5690c7c65, user_id=a1265b4b6c9f4f3fac7a85fc61a97cf0, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.292139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.368862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1261/2528] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:17 2022] PUT /identity/v3/projects/cb91fdd0ec954db297f44ad5690c7c65/users/a1265b4b6c9f4f3fac7a85fc61a97cf0/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:17.380917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.382005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.382005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.426749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] MFA Rules not processed for user `a1265b4b6c9f4f3fac7a85fc61a97cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:17.427977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:17.471464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:17.494016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1268/2529] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:17.499000 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.545114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:17.562643 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb2fa-2c31-44c6-bf09-22615e0044e9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.568687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.569298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.569582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.569854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.570531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.572534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.730734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1262/2530] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:17 2022] POST /identity/v3/projects => generated 391 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:17.734631 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.745214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd67afc-82e7-49bc-9c9f-6d54c1112b48 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.806383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.807005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.807312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.807584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.809169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.810183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.906803 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1269/2531] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:17 2022] POST /identity/v3/users => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:17.910649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.922421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735f18c-762e-482b-be9e-895e0ac33680 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.927524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.928150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.928434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.928708 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.929358 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.930979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:17.942296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1263/2532] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:17.946508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:17.958682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c4bc2-a0b3-4933-ae54-9a3e92fc5beb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:17.963670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:17.964366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:17.964651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:17.964929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] PATH_INFO: `/v3/projects/99da3f45e9f34f54b35b57b15b83c94f/users/c7421cfcb80442768329aeb8d78f390e/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:17.980285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99da3f45e9f34f54b35b57b15b83c94f, user_id=c7421cfcb80442768329aeb8d78f390e, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:17.983067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:18.065574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1270/2533] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:17 2022] PUT /identity/v3/projects/99da3f45e9f34f54b35b57b15b83c94f/users/c7421cfcb80442768329aeb8d78f390e/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:18.069526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:18.080334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f5d20-3017-4d83-88d6-7d979d07c4ed admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:18.146528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609846fc-831c-4fa9-ad5c-ce4fb7d44efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:18.147150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609846fc-831c-4fa9-ad5c-ce4fb7d44efa admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:18.147955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609846fc-831c-4fa9-ad5c-ce4fb7d44efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:18.148237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609846fc-831c-4fa9-ad5c-ce4fb7d44efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:18.148884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609846fc-831c-4fa9-ad5c-ce4fb7d44efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:18.150519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609846fc-831c-4fa9-ad5c-ce4fb7d44efa admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:18.161435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1264/2534] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:18.165751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:18.177216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b7402-506c-44ae-9c4b-ffa3c93adcb8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:18.182924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:18.183762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:18.184239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:18.184403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] PATH_INFO: `/v3/projects/99da3f45e9f34f54b35b57b15b83c94f/users/c7421cfcb80442768329aeb8d78f390e/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:18.186477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99da3f45e9f34f54b35b57b15b83c94f, user_id=c7421cfcb80442768329aeb8d78f390e, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:18.189890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:18.201517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1271/2535] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:18 2022] PUT /identity/v3/projects/99da3f45e9f34f54b35b57b15b83c94f/users/c7421cfcb80442768329aeb8d78f390e/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:18.211804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:18.212097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:18.212375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:18.242176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] MFA Rules not processed for user `c7421cfcb80442768329aeb8d78f390e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:18.243357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:18.316980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1265/2536] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:18.328095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:18.329831 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:18.391905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda5188-b21e-4d3b-9f1e-c0bad8084732 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:18.396360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:18.396986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:18.397270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:18.397547 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:18.407397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:18.408837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:18.418523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1272/2537] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:18.465538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:18.467430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:18.479615 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8667b1-f2f3-427f-8875-6af741db318c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:18.483419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:18.484029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:18.484303 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:18.484574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:18.519557 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:18.542333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:18.543810 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:18.553399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1266/2538] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:21.814045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:21.815494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:21.856876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:21.872057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a157c-bd1d-40d5-8ba2-51dd6aaca1d7 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:21.875688 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:21.876339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:21.876622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:21.876896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:21.878604 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:21.943134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:21.944649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a48c36-4f19-4549-b1ec-e0f0235f5053 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:21.954945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1273/2539] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:23.395949 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:23.408282 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00f7d-bb9c-45f0-9e1f-1fb4c1860a13 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:23.413578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:23.414605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:23.414969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:23.415335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] PATH_INFO: `/v3/users/a1265b4b6c9f4f3fac7a85fc61a97cf0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:23.417759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1265b4b6c9f4f3fac7a85fc61a97cf0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:23.418641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:23.621690 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1265b4b6c9f4f3fac7a85fc61a97cf0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:23.700739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1265b4b6c9f4f3fac7a85fc61a97cf0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:23.701553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1265b4b6c9f4f3fac7a85fc61a97cf0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:23.702248 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1265b4b6c9f4f3fac7a85fc61a97cf0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:23.713583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1265b4b6c9f4f3fac7a85fc61a97cf0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:23.739156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1267/2540] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:23 2022] DELETE /identity/v3/users/a1265b4b6c9f4f3fac7a85fc61a97cf0 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:23.743785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:23.744275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:23.744368 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:23.764466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:23.766338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:23.852337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1274/2541] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:23.862137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:23.863770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:23.929734 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e286d68-a09d-4f56-b7c8-be09e9e5df77 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:23.934285 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:23.935184 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:23.935620 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:23.936046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:23.955782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:23.963379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:23.988350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1268/2542] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:24.617509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:24.619013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:24.646367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b868f-13c1-4bad-ad5b-97ef99d5cf96 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:24.693619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:24.693619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:24.693619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:24.693619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] PATH_INFO: `/v3/projects/cb91fdd0ec954db297f44ad5690c7c65` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:24.698188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb91fdd0ec954db297f44ad5690c7c65)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:24.702169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:24.838657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb91fdd0ec954db297f44ad5690c7c65', 'request_id': 'req-8017e513-d6da-45e3-9588-e30d256376a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:24.850354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb91fdd0ec954db297f44ad5690c7c65', 'request_id': 'req-8017e513-d6da-45e3-9588-e30d256376a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:24.927325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb91fdd0ec954db297f44ad5690c7c65', 'request_id': 'req-8017e513-d6da-45e3-9588-e30d256376a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:24.948050 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] The token cache is being invalidate because project cb91fdd0ec954db297f44ad5690c7c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:24.948446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:24.949214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:24.951086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1275/2543] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:24 2022] DELETE /identity/v3/projects/cb91fdd0ec954db297f44ad5690c7c65 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:24.957903 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:24.976612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82806332-f237-4481-a31d-a37a32437417 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:25.074173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:25.087846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:25.088673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:25.089056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:25.089515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] PATH_INFO: `/v3/users/c7421cfcb80442768329aeb8d78f390e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:25.090985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7421cfcb80442768329aeb8d78f390e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:25.093120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:25.309619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7421cfcb80442768329aeb8d78f390e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:25.421044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7421cfcb80442768329aeb8d78f390e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:25.424963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7421cfcb80442768329aeb8d78f390e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:25.424963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7421cfcb80442768329aeb8d78f390e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:25.440503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7421cfcb80442768329aeb8d78f390e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:25.472523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1269/2544] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:24 2022] DELETE /identity/v3/users/c7421cfcb80442768329aeb8d78f390e => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:26.066322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.067988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.083525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017e513-d6da-45e3-9588-e30d256376a4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.102654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:26.158705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.158705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.158705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.158705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] PATH_INFO: `/v3/projects/99da3f45e9f34f54b35b57b15b83c94f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.159154 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99da3f45e9f34f54b35b57b15b83c94f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.160621 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.161096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.162644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.236574 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764760d-d2e7-4cb6-bc85-85a4326b30ea admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.241619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.243871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.243871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.243871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.244925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.256021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99da3f45e9f34f54b35b57b15b83c94f', 'request_id': 'req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.257272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.258841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.264591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99da3f45e9f34f54b35b57b15b83c94f', 'request_id': 'req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.270801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1270/2545] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:26.275258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.276843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.290829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d009-0c37-4467-8cf2-16283e4d0b67 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.343981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99da3f45e9f34f54b35b57b15b83c94f', 'request_id': 'req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.355766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] The token cache is being invalidate because project 99da3f45e9f34f54b35b57b15b83c94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:26.356178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.357082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d50ef9d6-5d5f-4a04-8a99-2c20a919f965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.358270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.358606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1276/2546] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:26 2022] DELETE /identity/v3/projects/99da3f45e9f34f54b35b57b15b83c94f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:26.359007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.359423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.359707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] PATH_INFO: `/v3/users/c069999293a847cdb11650045e22576c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.360830 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c069999293a847cdb11650045e22576c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.362865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.363390 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a096c1-c2b8-4167-a8a6-8fbf0be31e11 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.363684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a096c1-c2b8-4167-a8a6-8fbf0be31e11 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.363988 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a096c1-c2b8-4167-a8a6-8fbf0be31e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.364477 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:26.383973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d4a096c1-c2b8-4167-a8a6-8fbf0be31e11 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:26.385505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d4a096c1-c2b8-4167-a8a6-8fbf0be31e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.405444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1277/2547] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:26.413158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.413446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.417389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.433697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:26.434902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.443705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1278/2548] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:26.447677 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.456394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592a661-92cd-42aa-92dc-475668c8b71b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.461061 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.461637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.461918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.462188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.462822 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:26.463129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.466076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.478257 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1279/2549] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:23:26.482574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.495123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb073d8-eee0-4eb1-9869-ee4bef3c0b64 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.500624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.501236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.501520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.501792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.502425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.504793 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.587815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c069999293a847cdb11650045e22576c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.607578 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1280/2550] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:26 2022] POST /identity/v3/projects => generated 389 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:26.608587 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.654808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c069999293a847cdb11650045e22576c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.655577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c069999293a847cdb11650045e22576c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.656329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c069999293a847cdb11650045e22576c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.663649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c069999293a847cdb11650045e22576c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:26.669399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284460b-fbda-4ef3-8f7d-b817c09293e3 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.674694 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.675366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.675656 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.675941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.676605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.678693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.681785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1271/2551] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:26 2022] DELETE /identity/v3/users/c069999293a847cdb11650045e22576c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:26.797295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1281/2552] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:26 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:26.801408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.802800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.872384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:26.881159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08e791-e883-40dd-9cc1-6cd2035a0a67 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.885988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.886607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.886890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.887189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.887839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.889445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.899580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1272/2553] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:26.906046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.920947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d547e-e0bb-4616-8cd1-157dff0af5f2 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.926585 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.928486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.928486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.928486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.928486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.930056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.940559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1282/2554] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:26.945112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.956134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0162c-6fbb-470f-be62-ec08b72f3078 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.957314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:26.959642 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:26.961299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa1c5-bc64-4527-9fb1-33bf891c5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.962367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa1c5-bc64-4527-9fb1-33bf891c5190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.962660 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa1c5-bc64-4527-9fb1-33bf891c5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.962938 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa1c5-bc64-4527-9fb1-33bf891c5190 admin admin] PATH_INFO: `/v3/projects/b03c9423a2544025a4162db47341086b/users/d190063ad8b14ba7a1ea47bbaf16b5a4/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.980473 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0c761-a764-44ef-81b6-9435b5bf86d8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:26.981749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa1c5-bc64-4527-9fb1-33bf891c5190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03c9423a2544025a4162db47341086b, user_id=d190063ad8b14ba7a1ea47bbaf16b5a4, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.985860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa1c5-bc64-4527-9fb1-33bf891c5190 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:26.987532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:26.989458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:26.989923 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:26.990423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] PATH_INFO: `/v3/projects/b45075d7ec024b3a907d11225942fbee` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:26.991685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b45075d7ec024b3a907d11225942fbee)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:26.994042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.061404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1273/2555] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:26 2022] PUT /identity/v3/projects/b03c9423a2544025a4162db47341086b/users/d190063ad8b14ba7a1ea47bbaf16b5a4/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:27.072954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.073246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.073799 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.104272 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] MFA Rules not processed for user `d190063ad8b14ba7a1ea47bbaf16b5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:27.105511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:27.158613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b45075d7ec024b3a907d11225942fbee', 'request_id': 'req-fc6f7781-c300-4608-a116-14b58efdcae2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:27.165279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b45075d7ec024b3a907d11225942fbee', 'request_id': 'req-fc6f7781-c300-4608-a116-14b58efdcae2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:27.169354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1274/2556] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:27.176828 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.221179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b45075d7ec024b3a907d11225942fbee', 'request_id': 'req-fc6f7781-c300-4608-a116-14b58efdcae2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:27.232740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] The token cache is being invalidate because project b45075d7ec024b3a907d11225942fbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:27.233086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:27.233983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6f7781-c300-4608-a116-14b58efdcae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:27.235719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1283/2557] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:26 2022] DELETE /identity/v3/projects/b45075d7ec024b3a907d11225942fbee => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:27.240508 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d9f51-b460-4d37-95ff-6d86a891fd95 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.240966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d9f51-b460-4d37-95ff-6d86a891fd95 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.241512 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d9f51-b460-4d37-95ff-6d86a891fd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.248647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fe820-a05d-4dd1-811c-64a5b57ed88f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.251153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:27.264776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-291d9f51-b460-4d37-95ff-6d86a891fd95 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:27.266534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-291d9f51-b460-4d37-95ff-6d86a891fd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:27.318386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.319020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.319413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.319739 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.320406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.322426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.346370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1284/2558] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:27.354144 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.354429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.354703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.373864 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:27.378413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:27.384276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1285/2559] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:27.388550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.398370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4ff61-4615-43ca-81e0-1a09c6a059eb None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.403486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.404079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.404362 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.404648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.405356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:27.405666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.407974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.424366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1286/2560] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:23:27.424366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.433096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377f24d-931f-4f2c-bf18-580c8a19fa12 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.437729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.438288 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.438575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.438850 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.439510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.441388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.454957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1275/2561] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:27.456162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.457798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:27.523270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904734c8-61ac-4273-9151-aca33cb00c92 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.529779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.533309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1287/2562] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:27.533566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.533566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.533566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.533566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.533566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.535042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.597554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac174c-a42e-430d-9e42-4cb16852fb26 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.602110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:27.603950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.604560 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.604848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.605120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.605754 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.607709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.634183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1276/2563] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:27.636397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.646153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18a647-0ba8-4210-ab0e-9442e0bbe7e1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.650845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.651718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.652002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.652280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.652914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.654501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.665414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1277/2564] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:27.669332 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.679419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94819bf1-904b-498f-a8da-5af976f749f4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.684018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.684621 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.684904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.685187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] PATH_INFO: `/v3/projects/01b684254535485fbf6cbb1a610d2ec1/users/9025dc4b57bc41009af6eab9c7c5940a/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.699450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b684254535485fbf6cbb1a610d2ec1, user_id=9025dc4b57bc41009af6eab9c7c5940a, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.702073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.707121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1288/2565] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:27.708112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.717758 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8b80-2fdd-485d-ada6-f486db685d4a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.723028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.723653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.723937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.724214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.724868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.726543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.737253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1289/2566] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:27.741369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.755070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5e601-2912-40a3-bba3-7000d63ac544 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.760977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.761584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.761872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.762151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.762796 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.764627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.777444 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1290/2567] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:27.781950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.791638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64945b6-4cf9-453f-bf93-511a1451dd81 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.795284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1278/2568] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:27 2022] PUT /identity/v3/projects/01b684254535485fbf6cbb1a610d2ec1/users/9025dc4b57bc41009af6eab9c7c5940a/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:27.800187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.854155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e9674-db6b-46f5-b0c2-b29f38da7632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.854903 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9674-db6b-46f5-b0c2-b29f38da7632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.855202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9674-db6b-46f5-b0c2-b29f38da7632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.855478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9674-db6b-46f5-b0c2-b29f38da7632 admin admin] PATH_INFO: `/v3/projects/4436a7e677ac44f0bcc8f03b18c3d0ed/users/5746696d7afd4f17a3bda34cd2f42838/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.863316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730f43-8ea8-4988-8b99-8d0622eff339 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.868329 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.868942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.869223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.869498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.870183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.871485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e9674-db6b-46f5-b0c2-b29f38da7632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4436a7e677ac44f0bcc8f03b18c3d0ed, user_id=5746696d7afd4f17a3bda34cd2f42838, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.871773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.874334 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e9674-db6b-46f5-b0c2-b29f38da7632 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.876070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:27.883893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1279/2569] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:27.888220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:27.897910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aed2a8-715f-4766-9645-42b12511a65b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:27.902758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43259db-d6fe-40b9-a78a-b965b5f2fb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:27.903443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43259db-d6fe-40b9-a78a-b965b5f2fb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.907988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43259db-d6fe-40b9-a78a-b965b5f2fb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.907988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43259db-d6fe-40b9-a78a-b965b5f2fb95 admin admin] PATH_INFO: `/v3/projects/01b684254535485fbf6cbb1a610d2ec1/users/9025dc4b57bc41009af6eab9c7c5940a/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.907988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43259db-d6fe-40b9-a78a-b965b5f2fb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b684254535485fbf6cbb1a610d2ec1, user_id=9025dc4b57bc41009af6eab9c7c5940a, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:27.909082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43259db-d6fe-40b9-a78a-b965b5f2fb95 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:27.918924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1280/2570] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:27 2022] PUT /identity/v3/projects/01b684254535485fbf6cbb1a610d2ec1/users/9025dc4b57bc41009af6eab9c7c5940a/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:27.934584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:27.934584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:27.934584 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:27.957605 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] MFA Rules not processed for user `9025dc4b57bc41009af6eab9c7c5940a`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:27.958694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:27.987308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1291/2571] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:27 2022] PUT /identity/v3/projects/4436a7e677ac44f0bcc8f03b18c3d0ed/users/5746696d7afd4f17a3bda34cd2f42838/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:27.997718 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:28.002199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:28.002199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:28.027568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] MFA Rules not processed for user `5746696d7afd4f17a3bda34cd2f42838`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:28.029358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:28.033340 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1281/2572] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:28.044044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:28.045449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:28.085593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:28.098438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:28.099314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1292/2573] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:28.107595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f53d1-cc9c-4a07-bfcb-119dec080dc5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:28.110737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:28.112152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:28.157416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5c28b-3350-4a65-88ec-d66631202f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:28.158030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c28b-3350-4a65-88ec-d66631202f4e service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:28.158308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c28b-3350-4a65-88ec-d66631202f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:28.158579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c28b-3350-4a65-88ec-d66631202f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:28.191562 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25703ce2-73ba-4a5c-b528-06f32bca404d None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:28.194440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:28.195065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:28.195374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:28.195654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:28.204584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:28.206170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:28.215250 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1293/2574] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:28.224914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5c28b-3350-4a65-88ec-d66631202f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:28.226373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5c28b-3350-4a65-88ec-d66631202f4e service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:28.236243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1282/2575] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:28.382921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:28.446287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff63748-65b4-47d2-8189-9f23e3e580b1 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:28.450863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:28.451515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:28.451801 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:28.452078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] PATH_INFO: `/v3/users/d190063ad8b14ba7a1ea47bbaf16b5a4` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:28.453974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d190063ad8b14ba7a1ea47bbaf16b5a4)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:28.455927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:28.611908 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd190063ad8b14ba7a1ea47bbaf16b5a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:28.683096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd190063ad8b14ba7a1ea47bbaf16b5a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:28.683771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd190063ad8b14ba7a1ea47bbaf16b5a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:28.684386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd190063ad8b14ba7a1ea47bbaf16b5a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:28.692920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd190063ad8b14ba7a1ea47bbaf16b5a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:28.705856 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:28.710941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1294/2576] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:28 2022] DELETE /identity/v3/users/d190063ad8b14ba7a1ea47bbaf16b5a4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:28.715677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:28.715973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:28.716253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:28.739535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:28.740769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:28.806458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1283/2577] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:28.817685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:28.819442 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:28.880624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6a2e7-bef7-4124-bf4f-bdad442d4d47 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:28.884446 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:28.885052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:28.885331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:28.885685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:28.895745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:28.898296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:28.907768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1295/2578] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:29.320149 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:29.321641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:29.336260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebfcd1-3b78-4f06-be21-ca811dbd9b17 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:29.341447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:29.342258 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:29.342543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:29.342815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] PATH_INFO: `/v3/projects/b03c9423a2544025a4162db47341086b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:29.344798 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03c9423a2544025a4162db47341086b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:29.347238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:29.469820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:29.478607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03c9423a2544025a4162db47341086b', 'request_id': 'req-ac22f926-d55c-40f0-a832-641bb08172d8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.484779 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03c9423a2544025a4162db47341086b', 'request_id': 'req-ac22f926-d55c-40f0-a832-641bb08172d8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.554966 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03c9423a2544025a4162db47341086b', 'request_id': 'req-ac22f926-d55c-40f0-a832-641bb08172d8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.565509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] The token cache is being invalidate because project b03c9423a2544025a4162db47341086b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:29.565806 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.566433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.567960 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1284/2579] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:29 2022] DELETE /identity/v3/projects/b03c9423a2544025a4162db47341086b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:29.572057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:29.582298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eea33-a9c0-42ca-8ae4-f4d0cad1b527 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:29.638331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:29.638955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:29.639343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:29.639622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] PATH_INFO: `/v3/users/9025dc4b57bc41009af6eab9c7c5940a` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:29.641226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9025dc4b57bc41009af6eab9c7c5940a)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:29.643195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:29.855110 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9025dc4b57bc41009af6eab9c7c5940a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.932157 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9025dc4b57bc41009af6eab9c7c5940a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.932805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9025dc4b57bc41009af6eab9c7c5940a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.933415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9025dc4b57bc41009af6eab9c7c5940a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.941441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-38e5fc54-66b8-49dc-b60e-3a905d91a43c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9025dc4b57bc41009af6eab9c7c5940a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:29.960331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1296/2580] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:29 2022] DELETE /identity/v3/users/9025dc4b57bc41009af6eab9c7c5940a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:30.405018 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:30.406430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:30.421314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22f926-d55c-40f0-a832-641bb08172d8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:30.478404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:30.479028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:30.479352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:30.479630 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] PATH_INFO: `/v3/projects/01b684254535485fbf6cbb1a610d2ec1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:30.480733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b684254535485fbf6cbb1a610d2ec1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:30.482723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:30.610379 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b684254535485fbf6cbb1a610d2ec1', 'request_id': 'req-961f9907-619f-41b2-a4e2-72d0893585f2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:30.616636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b684254535485fbf6cbb1a610d2ec1', 'request_id': 'req-961f9907-619f-41b2-a4e2-72d0893585f2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:30.711638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b684254535485fbf6cbb1a610d2ec1', 'request_id': 'req-961f9907-619f-41b2-a4e2-72d0893585f2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:30.722827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] The token cache is being invalidate because project 01b684254535485fbf6cbb1a610d2ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:30.723121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:30.723771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-961f9907-619f-41b2-a4e2-72d0893585f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:30.725375 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1285/2581] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:30 2022] DELETE /identity/v3/projects/01b684254535485fbf6cbb1a610d2ec1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:30.730053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:30.730552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:30.730651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:30.750719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:30.751973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:30.764865 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:30.811531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1297/2582] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:30.820354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:30.820645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:30.820924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:30.840836 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:30.842087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:30.850575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1286/2583] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:30.854873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:30.865311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674958b9-d2bb-46ea-afcb-b5e6dd5e103b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:30.870121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:30.870782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:30.871066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:30.871412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:30.872070 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:30.872376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:30.874260 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:30.887302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1298/2584] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:30.891227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:30.900493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6486d37-3aad-44e4-b6ea-cd851da3969a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:30.905541 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:30.906777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:30.906905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:30.907231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:30.907862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:30.909749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:30.970236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1287/2585] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:30 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:30.974410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:30.984693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e58bf-f0c2-4d85-b58a-2c8830d976dc admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.039792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.040377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.040654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.040926 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.041548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.043457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.116776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1299/2586] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:30 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:31.120780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.130986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9853f6d-5909-4eed-8ea4-53aeb640adad admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.135698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.136281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.136905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.136905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.137445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.139023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.149740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1288/2587] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:31.153966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.165245 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456e39e-cdfa-40f7-9f1b-964afef04b7e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.169955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.170559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.170841 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.171124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.171839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.173425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.185343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1300/2588] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:31.189499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.200827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2919fee-7238-4d0a-bc90-5b8222bf3cc8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.206060 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96357bf4-5b05-44a6-afd5-34d51abb6b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.206705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357bf4-5b05-44a6-afd5-34d51abb6b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.206991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357bf4-5b05-44a6-afd5-34d51abb6b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.207351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357bf4-5b05-44a6-afd5-34d51abb6b1d admin admin] PATH_INFO: `/v3/projects/0598de65e2614fc1b17ae64cf3fbc18d/users/cc002513abd948d8a6bb5c500d1fc140/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.220928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:31.223931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96357bf4-5b05-44a6-afd5-34d51abb6b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0598de65e2614fc1b17ae64cf3fbc18d, user_id=cc002513abd948d8a6bb5c500d1fc140, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.226597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96357bf4-5b05-44a6-afd5-34d51abb6b1d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.248503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.250287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.308448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb9d8b-4405-490e-9a16-c633054d0976 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.311761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1289/2589] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:31 2022] PUT /identity/v3/projects/0598de65e2614fc1b17ae64cf3fbc18d/users/cc002513abd948d8a6bb5c500d1fc140/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:31.322497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.322867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.323239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.351399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:31.353860 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] MFA Rules not processed for user `cc002513abd948d8a6bb5c500d1fc140`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:31.355090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.361155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.361837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.362252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.362532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.364703 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.436165 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1290/2590] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:31.438734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.440279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.440441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.441914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.451284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1301/2591] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:31.452087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.456116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb3bf8-b6f5-4244-8313-d06497c4715d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.461284 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.462226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83288c21-577f-4523-aba7-753ed235b74c service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.462638 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.463103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.463612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] PATH_INFO: `/v3/users/5746696d7afd4f17a3bda34cd2f42838` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.464975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5746696d7afd4f17a3bda34cd2f42838)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.467138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.468625 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.469262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.469554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.469830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.470515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.472422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.530870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1302/2592] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:31 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:31.534942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.598616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4d6c1-3693-4fb3-9b8d-c8ebbb1a60b4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.605096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.605713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.606053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.606274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.608877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.608877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.626014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5746696d7afd4f17a3bda34cd2f42838'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:31.692520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1303/2593] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:31 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:31.693589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.703565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89fbf-4f36-4588-93ee-aa6b2c0a81b4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.708279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.708879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.709159 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.709430 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.710186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.711498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5746696d7afd4f17a3bda34cd2f42838'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:31.711826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.712350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5746696d7afd4f17a3bda34cd2f42838'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:31.713268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5746696d7afd4f17a3bda34cd2f42838'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:31.722239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8e67f052-c3e3-4614-b10a-3e404848ab8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5746696d7afd4f17a3bda34cd2f42838'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:31.724294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1304/2594] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:31.730376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.735009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.748186 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c252c5f-7e64-467a-99d6-092c6c987dac admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.751444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1291/2595] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:31 2022] DELETE /identity/v3/users/5746696d7afd4f17a3bda34cd2f42838 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:31.753830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.754568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.755037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.755509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] PATH_INFO: `/v3/projects/d7be1d245255486499b384c0a25bace4/users/c132f44166884723913c7eb32247a28d/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.756139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.756434 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.756716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.773003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7be1d245255486499b384c0a25bace4, user_id=c132f44166884723913c7eb32247a28d, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.776270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.782871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:31.784153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.833148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:31.845856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1292/2596] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:31.851195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1305/2597] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:31 2022] PUT /identity/v3/projects/d7be1d245255486499b384c0a25bace4/users/c132f44166884723913c7eb32247a28d/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:31.854879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.856445 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.858532 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:31.865945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801076c-f825-418c-86bd-12dd01acb703 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.892098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:31.926253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d53236-f724-4d94-87c6-71574f006a67 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.928762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.929551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.929837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.930113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.930510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20179fea-c333-440d-b6d7-af6d3384a141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.930764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.931928 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179fea-c333-440d-b6d7-af6d3384a141 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.932315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179fea-c333-440d-b6d7-af6d3384a141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.932433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.932774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179fea-c333-440d-b6d7-af6d3384a141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.946633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1293/2598] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:31.948104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:31.948966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20179fea-c333-440d-b6d7-af6d3384a141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.950529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20179fea-c333-440d-b6d7-af6d3384a141 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.960673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06a1d2-9d8b-4f40-852e-ef61646870bf admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:31.961641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1306/2599] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:31.967736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:31.968697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:31.969126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:31.969546 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] PATH_INFO: `/v3/projects/d7be1d245255486499b384c0a25bace4/users/c132f44166884723913c7eb32247a28d/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:31.971736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7be1d245255486499b384c0a25bace4, user_id=c132f44166884723913c7eb32247a28d, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:31.974528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:31.989609 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1294/2600] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:31 2022] PUT /identity/v3/projects/d7be1d245255486499b384c0a25bace4/users/c132f44166884723913c7eb32247a28d/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:32.001561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.002246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.003119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.047269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] MFA Rules not processed for user `c132f44166884723913c7eb32247a28d`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:32.049022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:32.129161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1307/2601] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:32.143534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:32.145260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:32.210881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208dd7-c171-43f8-8fb5-4b4c03e61451 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:32.214714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:32.215373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.215658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.215929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.217764 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:32.287300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:32.288782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25476e-c265-4eec-8215-d914b3bf9046 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:32.299199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1295/2602] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:32.401658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:32.402966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:32.476725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4d8ca-2d2b-448f-bb7b-cbaedea71877 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:32.481752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:32.482308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.482580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.482851 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] PATH_INFO: `/v3/projects/4436a7e677ac44f0bcc8f03b18c3d0ed` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.484030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4436a7e677ac44f0bcc8f03b18c3d0ed)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:32.485949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:32.599600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4436a7e677ac44f0bcc8f03b18c3d0ed', 'request_id': 'req-f848bae3-457b-42c2-9b26-d2d1edcc8c80'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:32.605768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4436a7e677ac44f0bcc8f03b18c3d0ed', 'request_id': 'req-f848bae3-457b-42c2-9b26-d2d1edcc8c80'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:32.607453 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:32.699576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4436a7e677ac44f0bcc8f03b18c3d0ed', 'request_id': 'req-f848bae3-457b-42c2-9b26-d2d1edcc8c80'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:32.714003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] The token cache is being invalidate because project 4436a7e677ac44f0bcc8f03b18c3d0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:32.714327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:32.714972 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f848bae3-457b-42c2-9b26-d2d1edcc8c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:32.716565 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1308/2603] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:32 2022] DELETE /identity/v3/projects/4436a7e677ac44f0bcc8f03b18c3d0ed => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:32.722127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.722640 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.722845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.748602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:32.750051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:32.784072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:32.825520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1296/2604] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:32.835399 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.835917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.836007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.860206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:32.861555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:32.870431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1309/2605] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:32.875743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:32.886333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c517d5-c08c-4ad2-bf28-d3df87fbe21b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:32.891601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:32.892204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.892480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.892749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.893393 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:32.893693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:32.895324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:32.908688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1297/2606] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:32.913202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:32.922600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110f507-a135-48dd-ba4a-06f5f76663ea None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:32.927392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:32.927994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:32.928268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:32.928533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:32.929168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:32.931041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.050261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1310/2607] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:32 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:33.051702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.063591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35311e9c-0187-4b31-b9dc-2b8e6e0ddd9f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.126941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.138823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.138823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.138823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.138823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.138823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.219210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1298/2608] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:33 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:33.230918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.242813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf77fd-4f91-4ccb-8c38-97fc026f8c59 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.248231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.249102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.249254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.249538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.250989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.252760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.267358 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1311/2609] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:33.271243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.285984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d2fb6-0cdb-45d5-89ee-b1d372103525 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.291519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4989d6-87f6-46e6-a610-6d9e4b0ffb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.292198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4989d6-87f6-46e6-a610-6d9e4b0ffb28 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.292481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4989d6-87f6-46e6-a610-6d9e4b0ffb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.292751 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4989d6-87f6-46e6-a610-6d9e4b0ffb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.293442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4989d6-87f6-46e6-a610-6d9e4b0ffb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.295077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4989d6-87f6-46e6-a610-6d9e4b0ffb28 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.310883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1299/2610] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:33.313079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.324567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f3777-dc81-4d07-9eaa-4b47af2e6c2b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.329945 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.330616 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.330894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.331394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] PATH_INFO: `/v3/projects/75dba19cd411437a820b2b8c38b7437f/users/760dfa0e71e9476992153d71e8788a02/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.348331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75dba19cd411437a820b2b8c38b7437f, user_id=760dfa0e71e9476992153d71e8788a02, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.351124 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.450767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1312/2611] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:33 2022] PUT /identity/v3/projects/75dba19cd411437a820b2b8c38b7437f/users/760dfa0e71e9476992153d71e8788a02/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:33.454109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.454417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.454688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.484623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] MFA Rules not processed for user `760dfa0e71e9476992153d71e8788a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:33.485756 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:33.558425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1300/2612] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:33.559619 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.630635 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59479615-b0ac-458b-baf5-20e5ee193b75 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.636812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.637505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.637795 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.638071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.638720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.640833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.739129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1313/2613] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:33 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:33.743459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.757134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a72435-c0dd-403b-91bf-056e68143cff None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.789100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:33.824510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.825224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.825582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.825930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.826649 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.828665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.900288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1301/2614] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:33 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:33.904750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.915808 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f336f-3a1f-4dde-b180-499cbd5b1faa admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.920510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.921104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.921391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.921735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.922961 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.924622 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:33.939942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1314/2615] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:33.941422 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:33.954139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3d6bb-17aa-4005-bdb9-b31f5daff4d6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:33.959130 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:33.959827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:33.960112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:33.960411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] PATH_INFO: `/v3/projects/34f215463b8847f1b4c865dc03f0c684/users/c4a9f7408e3940a9a4ad0f89cc15fb9c/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:33.976407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f215463b8847f1b4c865dc03f0c684, user_id=c4a9f7408e3940a9a4ad0f89cc15fb9c, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:33.979108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:34.055191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1302/2616] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:33 2022] PUT /identity/v3/projects/34f215463b8847f1b4c865dc03f0c684/users/c4a9f7408e3940a9a4ad0f89cc15fb9c/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:34.059382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:34.069608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2321b-2ffc-45bc-8826-d5f28814a7b9 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:34.125123 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:34.143210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af36bb0-1116-4e4b-8507-fe9d09276da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:34.143210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af36bb0-1116-4e4b-8507-fe9d09276da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:34.143210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af36bb0-1116-4e4b-8507-fe9d09276da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:34.143210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af36bb0-1116-4e4b-8507-fe9d09276da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:34.143704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af36bb0-1116-4e4b-8507-fe9d09276da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:34.144704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af36bb0-1116-4e4b-8507-fe9d09276da4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:34.159626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1315/2617] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:34.162665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:34.175655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9e9a-523a-4954-a7cd-d5bfce8869bf admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:34.185823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:34.185823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:34.185823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:34.185823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] PATH_INFO: `/v3/projects/34f215463b8847f1b4c865dc03f0c684/users/c4a9f7408e3940a9a4ad0f89cc15fb9c/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:34.188278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f215463b8847f1b4c865dc03f0c684, user_id=c4a9f7408e3940a9a4ad0f89cc15fb9c, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:34.191335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:34.213936 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1303/2618] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:34 2022] PUT /identity/v3/projects/34f215463b8847f1b4c865dc03f0c684/users/c4a9f7408e3940a9a4ad0f89cc15fb9c/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:34.223700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:34.223994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:34.224254 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:34.252027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] MFA Rules not processed for user `c4a9f7408e3940a9a4ad0f89cc15fb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:34.253440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:34.336978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1316/2619] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:34.349470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:34.351497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:34.420736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af79bb-a475-425f-852b-c1e55956fb40 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:34.424707 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:34.425327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:34.425601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:34.425872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:34.490053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:34.491559 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:34.502160 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1304/2620] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:35.688263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:35.689828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:35.755860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30aee1-c6aa-4ec2-8734-052ac3b9945f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:35.760441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:35.761018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:35.761302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:35.761596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] PATH_INFO: `/v3/users/90f07b101240464387f3fbf92c925ec7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:35.762701 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f07b101240464387f3fbf92c925ec7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:35.764665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:35.940963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f07b101240464387f3fbf92c925ec7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.023381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f07b101240464387f3fbf92c925ec7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.023974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f07b101240464387f3fbf92c925ec7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.024584 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f07b101240464387f3fbf92c925ec7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.032721 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc088dcd-cd06-499c-a1d7-ae5a6131f347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f07b101240464387f3fbf92c925ec7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.042826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:36.050909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1317/2621] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:35 2022] DELETE /identity/v3/users/90f07b101240464387f3fbf92c925ec7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:36.296113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:36.308858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cddbf-7660-4706-8c16-b038573dc6ba service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:36.352361 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:36.379376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:36.380012 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:36.380294 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:36.380573 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] PATH_INFO: `/v3/projects/c9aaa16702a5422b90578033420da9bd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:36.381723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9aaa16702a5422b90578033420da9bd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:36.383686 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:36.543932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9aaa16702a5422b90578033420da9bd', 'request_id': 'req-74ed1065-e026-4fa5-af6b-43c2228fbb65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.552419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9aaa16702a5422b90578033420da9bd', 'request_id': 'req-74ed1065-e026-4fa5-af6b-43c2228fbb65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.626169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9aaa16702a5422b90578033420da9bd', 'request_id': 'req-74ed1065-e026-4fa5-af6b-43c2228fbb65'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.639028 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] The token cache is being invalidate because project c9aaa16702a5422b90578033420da9bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:36.639440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.640745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-74ed1065-e026-4fa5-af6b-43c2228fbb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:36.642328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1305/2622] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:36 2022] DELETE /identity/v3/projects/c9aaa16702a5422b90578033420da9bd => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:36.647377 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:36.647668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:36.647947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:36.669434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:36.670643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:36.730738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1318/2623] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:36.738577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:36.738894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:36.739208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:36.762253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:36.763674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:36.773732 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1306/2624] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:36.778383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:36.788386 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2211021-d1e4-42f3-87de-c69c8863663e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:36.792889 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:36.793478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:36.793753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:36.798287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:36.798287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:36.798287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:36.798287 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:36.808369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1319/2625] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:36.813120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:36.822402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc41e6-d0fa-4d31-8097-6f76e80f11d2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:36.827262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:36.828539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:36.828818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:36.829094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:36.829730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:36.831657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:36.945871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1307/2626] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:36 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:36.950063 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:36.960002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfac4e4-a5fd-4d9e-97f4-6f9c0f711873 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:37.021391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:37.022041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.022321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.022637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.023355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:37.025246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:37.106246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1320/2627] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:36 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:37.110833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:37.121349 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ed5f8-94dc-45dd-93bf-56180e7e8751 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:37.126683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:37.127344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.127599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.127874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.128524 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:37.130106 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:37.140042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1308/2628] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:37.144013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:37.154269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d955d9-4de0-41dc-96f7-5c95647cb5cd admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:37.159612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cb1c6-6a69-4d64-8c54-6fb3df1a7fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:37.160545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cb1c6-6a69-4d64-8c54-6fb3df1a7fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.161010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cb1c6-6a69-4d64-8c54-6fb3df1a7fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.161472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cb1c6-6a69-4d64-8c54-6fb3df1a7fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.162558 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cb1c6-6a69-4d64-8c54-6fb3df1a7fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:37.164704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cb1c6-6a69-4d64-8c54-6fb3df1a7fa0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:37.175895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1321/2629] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:37.180115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:37.190818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb6d9e-dd94-4147-9944-e304e08626f0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:37.195703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:37.196397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.196680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.196955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] PATH_INFO: `/v3/projects/eec6632f73df41f0b93ab6cb261c19fc/users/509a923e45854118955f7b9925463ca2/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.211599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec6632f73df41f0b93ab6cb261c19fc, user_id=509a923e45854118955f7b9925463ca2, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:37.214315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:37.338092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1309/2630] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:37 2022] PUT /identity/v3/projects/eec6632f73df41f0b93ab6cb261c19fc/users/509a923e45854118955f7b9925463ca2/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:37.349290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.349576 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.349847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.377426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] MFA Rules not processed for user `509a923e45854118955f7b9925463ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:37.378657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:37.426263 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:37.443024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1322/2631] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:37.454466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:37.456213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:37.515346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:37.531890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c737-eb55-48dc-8f54-4ba68d2892c8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:37.537423 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:37.538022 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.538325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.538963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.547191 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:37.548757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:37.557533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1310/2632] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:37.723438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:37.724853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:37.786711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec70f5d-ad61-4e05-a064-d98b73a349af None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:37.790982 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:37.791702 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:37.792018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:37.792339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:37.794370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:37.865394 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:37.866780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:37.876156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1323/2633] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:40.326459 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:40.345759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097f4b-4a83-47d4-9ca7-7ff8198e3ff9 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:40.349390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:40.349986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:40.350269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:40.350542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:40.352308 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:40.451583 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:40.453127 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595c6f2-19a4-40c5-8afc-64de3a728e41 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:40.462730 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1311/2634] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:48.018129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=55 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:49.393509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:49.501762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e0e0-e518-4ff3-9a5b-6678142b17cd service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:49.512406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:49.513121 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:49.513293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:49.513592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] PATH_INFO: `/v3/users/509a923e45854118955f7b9925463ca2` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:49.515495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=509a923e45854118955f7b9925463ca2)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:49.521044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:49.708428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509a923e45854118955f7b9925463ca2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:49.756505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509a923e45854118955f7b9925463ca2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:49.757122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509a923e45854118955f7b9925463ca2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:49.757785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509a923e45854118955f7b9925463ca2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:49.773271 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509a923e45854118955f7b9925463ca2'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:49.782475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1324/2635] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:49 2022] DELETE /identity/v3/users/509a923e45854118955f7b9925463ca2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:49.786999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:49.787517 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:49.787627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:49.826915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:49.828173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:49.886023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1312/2636] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:49.895951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:49.897375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:49.939375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:49.965751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0078-6cb7-46c3-80b0-5f0365dc55ad admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:49.970208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:49.970875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:49.971158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:49.971460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:49.980520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:49.986166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:49.992028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1325/2637] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:50.316928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:50.318369 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:50.330883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b1017-c8e6-43e7-bd60-45116e1d5baf None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:50.335473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:50.336217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.336503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.336783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] PATH_INFO: `/v3/users/cc002513abd948d8a6bb5c500d1fc140` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.337908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc002513abd948d8a6bb5c500d1fc140)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:50.339858 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:50.356007 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:50.381019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:50.383932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:50.396937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4839b1-311d-4c69-ad5a-8bd234c81245 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:50.402332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:50.403043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.403353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.403626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] PATH_INFO: `/v3/projects/eec6632f73df41f0b93ab6cb261c19fc` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.404802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec6632f73df41f0b93ab6cb261c19fc)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:50.407284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:50.571866 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc002513abd948d8a6bb5c500d1fc140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.572105 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec6632f73df41f0b93ab6cb261c19fc', 'request_id': 'req-47768523-4f83-4fef-b421-fc9dc7fc7249'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.578947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec6632f73df41f0b93ab6cb261c19fc', 'request_id': 'req-47768523-4f83-4fef-b421-fc9dc7fc7249'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.630551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc002513abd948d8a6bb5c500d1fc140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.631138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc002513abd948d8a6bb5c500d1fc140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.631785 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc002513abd948d8a6bb5c500d1fc140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.639073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e8e428e-ea9f-4b30-a7f7-d961803eb776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc002513abd948d8a6bb5c500d1fc140'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.658725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1313/2638] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:50 2022] DELETE /identity/v3/users/cc002513abd948d8a6bb5c500d1fc140 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:50.663035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.670678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.670678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.682035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:50.683260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:50.687842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec6632f73df41f0b93ab6cb261c19fc', 'request_id': 'req-47768523-4f83-4fef-b421-fc9dc7fc7249'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.698918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] The token cache is being invalidate because project eec6632f73df41f0b93ab6cb261c19fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:50.699248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.704338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-47768523-4f83-4fef-b421-fc9dc7fc7249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:50.704338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1326/2639] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:50 2022] DELETE /identity/v3/projects/eec6632f73df41f0b93ab6cb261c19fc => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:50.706878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350b6d6-4a62-4bd4-9d9a-feea543af167 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.707208 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350b6d6-4a62-4bd4-9d9a-feea543af167 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.707487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350b6d6-4a62-4bd4-9d9a-feea543af167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.745311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1314/2640] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:50.755417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:50.756941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:50.770812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e350b6d6-4a62-4bd4-9d9a-feea543af167 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:50.772060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e350b6d6-4a62-4bd4-9d9a-feea543af167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:50.781413 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1327/2641] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:50.790441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.790736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.791013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.810236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:50.812126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:50.816225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af184ffb-3ab5-4e38-9463-750a4e4d16a8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:50.820070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:50.820568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1328/2642] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:50.820688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.820954 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.821231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.824475 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:50.831722 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:50.833169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:50.835710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f4ed-1b11-4f7c-86d0-c4223415df86 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:50.839496 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:50.840111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.840381 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.840651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.841293 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:50.841606 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:50.843238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:50.843757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1315/2643] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:50.855284 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1329/2644] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:23:50.859714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:50.869364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb47c3-89dd-41a4-a791-c224af5c5604 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:50.874591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:50.875213 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:50.875498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:50.875774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:50.876526 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:50.878454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:50.980937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1316/2645] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:50 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:50.985327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:50.995363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3384e4-759f-45cb-8ffa-189c9da77e27 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.053722 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.054338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.054623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.054906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.055581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.057497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.068684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:51.165626 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1330/2646] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:50 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:51.166905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.178015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf700-c0e9-4093-b1e6-0211962905e5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.183474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.184126 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.184403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.184675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.185320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.187670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.197783 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1317/2647] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:51.204169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.214295 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1cbe1-d5c3-4a5e-b7dc-24424727ff63 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.219045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.219676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.219968 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.220246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.220887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.222477 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.233661 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1331/2648] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:51.237910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.244528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:51.248520 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255b5a2-c113-4b3a-b802-f5b1585f3aaf admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.254242 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a641-c0d4-4cf5-a0a2-9455cf452f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.255005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a641-c0d4-4cf5-a0a2-9455cf452f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.257223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a641-c0d4-4cf5-a0a2-9455cf452f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.257223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a641-c0d4-4cf5-a0a2-9455cf452f2a admin admin] PATH_INFO: `/v3/projects/2b7a936ec9cc4ee69ab4159ad5b0e7df/users/9873d40ae2b147b1ae1ec1aab2edf85e/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.262343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.264243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:51.277805 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f633b-7f04-431c-993c-a5f973fdd028 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.282827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.283494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.283772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.284045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] PATH_INFO: `/v3/projects/0598de65e2614fc1b17ae64cf3fbc18d` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.285169 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0598de65e2614fc1b17ae64cf3fbc18d)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.287313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.287577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7a641-c0d4-4cf5-a0a2-9455cf452f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7a936ec9cc4ee69ab4159ad5b0e7df, user_id=9873d40ae2b147b1ae1ec1aab2edf85e, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.290415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7a641-c0d4-4cf5-a0a2-9455cf452f2a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.375144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1318/2649] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:51 2022] PUT /identity/v3/projects/2b7a936ec9cc4ee69ab4159ad5b0e7df/users/9873d40ae2b147b1ae1ec1aab2edf85e/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:51.385607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.385891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.386168 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.415636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] MFA Rules not processed for user `9873d40ae2b147b1ae1ec1aab2edf85e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:51.416878 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:51.451149 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0598de65e2614fc1b17ae64cf3fbc18d', 'request_id': 'req-c43a83a5-98d7-4232-9d06-81c75e6f16da'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.457469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0598de65e2614fc1b17ae64cf3fbc18d', 'request_id': 'req-c43a83a5-98d7-4232-9d06-81c75e6f16da'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.481009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1319/2650] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:51.493067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.494391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:51.574126 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0598de65e2614fc1b17ae64cf3fbc18d', 'request_id': 'req-c43a83a5-98d7-4232-9d06-81c75e6f16da'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.586674 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] The token cache is being invalidate because project 0598de65e2614fc1b17ae64cf3fbc18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:51.587052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.587837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.592224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1332/2651] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:51 2022] DELETE /identity/v3/projects/0598de65e2614fc1b17ae64cf3fbc18d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:51.593704 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.595069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:51.599515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863b6ce-2451-4bf7-a899-ad0086c18adf None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.640488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:51.659863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a83a5-98d7-4232-9d06-81c75e6f16da admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.664515 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.664674 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.665137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.665275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.665415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.665562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.665683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] PATH_INFO: `/v3/users/c132f44166884723913c7eb32247a28d` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.666010 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.666813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c132f44166884723913c7eb32247a28d)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.667817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:51.668748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.727628 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.729227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:51.738793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1320/2652] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:51.857265 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c132f44166884723913c7eb32247a28d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.946138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c132f44166884723913c7eb32247a28d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.946138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c132f44166884723913c7eb32247a28d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.946138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c132f44166884723913c7eb32247a28d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.954782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:51.956341 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:51.959010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c44ed7d-a575-4e2a-958f-0ee29f78e49b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c132f44166884723913c7eb32247a28d'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:51.963895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:51.970187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ec39-8d5a-4099-af34-7ff47e1a4819 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:51.975097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:51.975861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:51.976220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:51.976496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] PATH_INFO: `/v3/users/9873d40ae2b147b1ae1ec1aab2edf85e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:51.977767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9873d40ae2b147b1ae1ec1aab2edf85e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:51.978876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1333/2653] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:51 2022] DELETE /identity/v3/users/c132f44166884723913c7eb32247a28d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:51.981902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:52.223391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9873d40ae2b147b1ae1ec1aab2edf85e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.322606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9873d40ae2b147b1ae1ec1aab2edf85e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.323304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9873d40ae2b147b1ae1ec1aab2edf85e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.323928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9873d40ae2b147b1ae1ec1aab2edf85e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.332597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9873d40ae2b147b1ae1ec1aab2edf85e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.353092 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1321/2654] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:51 2022] DELETE /identity/v3/users/9873d40ae2b147b1ae1ec1aab2edf85e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:52.357601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.357888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.358195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.377696 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:52.378948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:52.420979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:52.422414 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:52.454357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1334/2655] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:52.464339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:52.465717 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:52.482443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:52.495556 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543573a-6363-49cb-9c5e-eb47c3dcd292 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:52.501493 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:52.502458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.502458 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.502733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] PATH_INFO: `/v3/projects/d7be1d245255486499b384c0a25bace4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.503905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7be1d245255486499b384c0a25bace4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:52.505840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:52.530712 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86199de-568d-451e-98a5-971511e5adc5 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:52.534732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf0861-f373-422f-a458-029cab3c46c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:52.535634 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf0861-f373-422f-a458-029cab3c46c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.536075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf0861-f373-422f-a458-029cab3c46c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.536548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf0861-f373-422f-a458-029cab3c46c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.546511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf0861-f373-422f-a458-029cab3c46c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:52.547920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf0861-f373-422f-a458-029cab3c46c0 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:52.558116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1335/2656] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:52.614904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7be1d245255486499b384c0a25bace4', 'request_id': 'req-eb7820b7-676c-4211-9be1-bfe3c7d69d69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.622821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7be1d245255486499b384c0a25bace4', 'request_id': 'req-eb7820b7-676c-4211-9be1-bfe3c7d69d69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.709081 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7be1d245255486499b384c0a25bace4', 'request_id': 'req-eb7820b7-676c-4211-9be1-bfe3c7d69d69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.722350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] The token cache is being invalidate because project d7be1d245255486499b384c0a25bace4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:52.722879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.723772 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb7820b7-676c-4211-9be1-bfe3c7d69d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:52.726082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1322/2657] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:52 2022] DELETE /identity/v3/projects/d7be1d245255486499b384c0a25bace4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:52.732543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.732777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.733048 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.755396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:52.756897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:52.820057 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1336/2658] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:52.885478 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.885765 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.886045 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.906114 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:52.909775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:52.915355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1323/2659] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:52.919586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:52.929646 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6749-18c4-4202-84a9-35fbbe8d7023 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:52.934239 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:52.934835 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.938443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.938443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.938443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:52.938443 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:52.938799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:52.949900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1337/2660] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:52.954304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:52.964020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05025e19-a37a-4455-af04-174d53c0c8e1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:52.968787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:52.969391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:52.969675 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:52.969950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:52.970587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:52.972587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:52.993449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:52.996027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:53.009178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98242c1-c273-4e67-9d3e-cfe3cb0197e8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.014111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.014772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.015474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.015474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] PATH_INFO: `/v3/projects/2b7a936ec9cc4ee69ab4159ad5b0e7df` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.016602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7a936ec9cc4ee69ab4159ad5b0e7df)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.018531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.031660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:53.050714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1324/2661] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:52 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:53.054790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.117087 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7a936ec9cc4ee69ab4159ad5b0e7df', 'request_id': 'req-f8f7a614-9c05-44ab-a8b3-b0701f073813'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:53.120125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de041f-0c7a-4516-94fe-45c45a651c4a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.124296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7a936ec9cc4ee69ab4159ad5b0e7df', 'request_id': 'req-f8f7a614-9c05-44ab-a8b3-b0701f073813'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:53.182776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.183416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.183700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.183972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.184600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.186515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.194417 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:53.202119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7a936ec9cc4ee69ab4159ad5b0e7df', 'request_id': 'req-f8f7a614-9c05-44ab-a8b3-b0701f073813'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:53.212871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] The token cache is being invalidate because project 2b7a936ec9cc4ee69ab4159ad5b0e7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:53.213161 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:53.213801 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-f8f7a614-9c05-44ab-a8b3-b0701f073813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:53.215338 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1338/2662] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:52 2022] DELETE /identity/v3/projects/2b7a936ec9cc4ee69ab4159ad5b0e7df => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:53.220482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1edec-65ff-40ad-8016-0f179584fd97 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.220765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1edec-65ff-40ad-8016-0f179584fd97 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.221037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1edec-65ff-40ad-8016-0f179584fd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.241363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-b6a1edec-65ff-40ad-8016-0f179584fd97 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:53.242319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-b6a1edec-65ff-40ad-8016-0f179584fd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:53.250275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1339/2663] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:53.257713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.257993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.262937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.277272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:53.278449 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:53.288258 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1340/2664] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:53.292561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.294595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1325/2665] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:53 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:53.299275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.301001 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:53.302744 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555762a-ccc5-47c0-8438-7e6d3855c4e2 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.307452 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.308049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.308360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.308633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.309306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:53.309768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.311385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.312834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42da0ca-5467-476e-85ac-8c94d25342ca admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.317769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.318376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.318657 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.318930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.319601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.321239 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.326424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1341/2666] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:23:53.327647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.332427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1326/2667] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:53.338294 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67090a-6e57-4298-ba9d-be4ba1e7afba admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.341148 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.343917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.344509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.344827 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.345097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.345733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.347665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.351227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829624-0e64-4d90-b827-d5a54f03545c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.355857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.362290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.362290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.362290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.362290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.362290 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.370396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1327/2668] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:53.374480 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.383915 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae96c74-80ab-408f-a50a-d3e22b25ab87 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.388606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5af7e-d669-439e-89b9-d9d8108f675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.389255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5af7e-d669-439e-89b9-d9d8108f675c admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.389564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5af7e-d669-439e-89b9-d9d8108f675c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.389840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5af7e-d669-439e-89b9-d9d8108f675c admin admin] PATH_INFO: `/v3/projects/d284c6051db249ab994f2b9e0073de99/users/fb87a36fc7e744a8887d3b2e11cb7473/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.406109 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5af7e-d669-439e-89b9-d9d8108f675c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d284c6051db249ab994f2b9e0073de99, user_id=fb87a36fc7e744a8887d3b2e11cb7473, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.430381 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5af7e-d669-439e-89b9-d9d8108f675c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.540942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1342/2669] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:53 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:53.544055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.617109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c01fa3-ee39-45c5-9137-1effc4b9c582 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.622441 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.623195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.623487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.623764 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.624425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.626518 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.723410 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1328/2670] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:53 2022] PUT /identity/v3/projects/d284c6051db249ab994f2b9e0073de99/users/fb87a36fc7e744a8887d3b2e11cb7473/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:53.734291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.734582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.734863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.761894 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1343/2671] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:53 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:53.763979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] MFA Rules not processed for user `fb87a36fc7e744a8887d3b2e11cb7473`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:53.765339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:53.767410 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.777752 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da775b65-8143-409f-a99e-18aaf89d0a63 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.821897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:53.840489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.841106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.841472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.841753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.842360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.844038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.849409 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1329/2672] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:53.859343 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1344/2673] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:53.861901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.863668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.867370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:53.877908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5619b-cfa1-45d2-87e3-bc14a9daf822 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.883229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.883918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.884274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.884624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.885380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.887144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.898027 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1330/2674] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:53.902677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.913945 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328dfa-930a-4ef3-9e9f-eb56a8c3463d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.920223 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7a37-9840-4457-8354-45fbea74b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.922121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7a37-9840-4457-8354-45fbea74b95c admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.922685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7a37-9840-4457-8354-45fbea74b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.923281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7a37-9840-4457-8354-45fbea74b95c admin admin] PATH_INFO: `/v3/projects/646eb6c170834f0ab978bd07582277ca/users/3fed882c541b47fb847bd28848db82fc/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.937932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0df62-f1d1-4ca1-aac9-ff9614173584 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:53.942104 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:53.943384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:53.943719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:53.943953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:53.944062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f7a37-9840-4457-8354-45fbea74b95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=646eb6c170834f0ab978bd07582277ca, user_id=3fed882c541b47fb847bd28848db82fc, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.946887 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f7a37-9840-4457-8354-45fbea74b95c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.957241 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:53.958777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:53.967598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1345/2675] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:53.986685 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:53.997243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec70f-f258-4b35-b936-e70e4def2fb3 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.002459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.006673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.006673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.006673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] PATH_INFO: `/v3/users/fb87a36fc7e744a8887d3b2e11cb7473` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.006673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb87a36fc7e744a8887d3b2e11cb7473)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.007028 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.032397 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1331/2676] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:53 2022] PUT /identity/v3/projects/646eb6c170834f0ab978bd07582277ca/users/3fed882c541b47fb847bd28848db82fc/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:54.044169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.044452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.044741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.073827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] MFA Rules not processed for user `3fed882c541b47fb847bd28848db82fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:54.074992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:54.121588 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:54.142770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1332/2677] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:54.147914 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.197177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb87a36fc7e744a8887d3b2e11cb7473'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:54.219716 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390dbfe-9e21-4bef-b344-17721cb5e4f6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.224790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.225407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.225672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.225991 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.226613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.228531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.266042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb87a36fc7e744a8887d3b2e11cb7473'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:54.267182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb87a36fc7e744a8887d3b2e11cb7473'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:54.267897 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb87a36fc7e744a8887d3b2e11cb7473'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:54.275821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-41d3406b-a1c5-4125-a9cb-01ff9f71f2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb87a36fc7e744a8887d3b2e11cb7473'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:54.293999 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1346/2678] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:23:53 2022] DELETE /identity/v3/users/fb87a36fc7e744a8887d3b2e11cb7473 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:54.298762 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.299054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.299361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.330343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1333/2679] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:54 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:23:54.331482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:54.332933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:54.334782 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.336687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:54.389591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:54.411534 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1347/2680] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:23:54.415890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3c8ba-e026-424d-9bf3-ad8b7a5a5127 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.419570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.421592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.421844 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:54.422321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.422614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.422893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.423600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.425562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.555363 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1334/2681] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:54 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:54.556902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.568709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a5c4-3a9f-4abf-bf51-8774a9b5aa6b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.573956 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.574641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.574926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.575225 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.575884 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.577606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.590348 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1335/2682] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:54.596595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.608084 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2b0c-e433-4185-a35b-4e0641ff578d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.613888 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.614551 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.614837 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.615198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] PATH_INFO: `/v3/projects/00b64e3bf200401bb5925faedbb2b276/users/6ffa47f98c5040c9a6805c3fdbbc9f91/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.622269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bfe34-91f0-4c68-b5a8-51ca2a36625b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.626131 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.630429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.630429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.630429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.633977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b64e3bf200401bb5925faedbb2b276, user_id=6ffa47f98c5040c9a6805c3fdbbc9f91, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.637030 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.638253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.640092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.650196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1348/2683] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:54.712863 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1336/2684] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:54 2022] PUT /identity/v3/projects/00b64e3bf200401bb5925faedbb2b276/users/6ffa47f98c5040c9a6805c3fdbbc9f91/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:54.716918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.727276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf104a-593f-42b9-84c4-45cd3f0b9c6c service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.786735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0210d9-8934-47f9-ba4f-ee2e80d34b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.787393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0210d9-8934-47f9-ba4f-ee2e80d34b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.787675 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0210d9-8934-47f9-ba4f-ee2e80d34b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.787939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0210d9-8934-47f9-ba4f-ee2e80d34b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.788570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0210d9-8934-47f9-ba4f-ee2e80d34b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.790167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0210d9-8934-47f9-ba4f-ee2e80d34b9b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.800550 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1349/2685] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:54.804848 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.815317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61178ce-d5d6-4883-9b06-bad52a23fcf3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:54.820005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:54.820668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.820964 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.821304 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] PATH_INFO: `/v3/projects/00b64e3bf200401bb5925faedbb2b276/users/6ffa47f98c5040c9a6805c3fdbbc9f91/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.823260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b64e3bf200401bb5925faedbb2b276, user_id=6ffa47f98c5040c9a6805c3fdbbc9f91, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:54.825874 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:54.838929 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1337/2686] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:54 2022] PUT /identity/v3/projects/00b64e3bf200401bb5925faedbb2b276/users/6ffa47f98c5040c9a6805c3fdbbc9f91/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:54.850229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:54.850450 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:54.850699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:54.881397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] MFA Rules not processed for user `6ffa47f98c5040c9a6805c3fdbbc9f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:54.882614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:54.959268 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1350/2687] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:54.973122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:54.975486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:54.998536 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:55.043269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181146d8-e46b-42ee-89b4-b43b94ea72e5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:55.047037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:55.047679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.047958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.048229 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.057999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:55.059530 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:55.069256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1338/2688] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:55.107256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:55.108766 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:55.141119 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:55.200551 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f1c40-4d64-4546-979a-f99627270a7c None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:55.206950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:55.207623 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.208143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.208432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] PATH_INFO: `/v3/projects/d284c6051db249ab994f2b9e0073de99` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.209599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d284c6051db249ab994f2b9e0073de99)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:55.211555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:55.290407 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:55.299218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ef814-ffe8-45ef-b673-804d040c7a4c service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:55.304985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:55.309344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.309344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.309344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.309344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:55.367121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:55.368507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94ad6-a804-49fb-9cac-3d2b09b72ad4 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:55.378255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1339/2689] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:23:55.399902 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd284c6051db249ab994f2b9e0073de99', 'request_id': 'req-dd813202-3568-48b2-90e7-72babf959d8e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:55.406609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd284c6051db249ab994f2b9e0073de99', 'request_id': 'req-dd813202-3568-48b2-90e7-72babf959d8e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:55.480116 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd284c6051db249ab994f2b9e0073de99', 'request_id': 'req-dd813202-3568-48b2-90e7-72babf959d8e'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:55.491005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] The token cache is being invalidate because project d284c6051db249ab994f2b9e0073de99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:23:55.491326 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:55.491947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-dd813202-3568-48b2-90e7-72babf959d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:23:55.493437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1351/2690] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:23:55 2022] DELETE /identity/v3/projects/d284c6051db249ab994f2b9e0073de99 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:23:55.498335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.499138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.499448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.519532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:55.521154 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:55.600717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1340/2691] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:55.608415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.608738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.609019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.629480 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:55.630637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:55.639238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1352/2692] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:55.647091 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:55.657512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c611-8689-4eb5-8df1-e50af37a6cc3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:55.662295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:55.662911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.663228 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.663506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.664180 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:23:55.664485 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:55.666071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:55.678670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1341/2693] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:23:55.683956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:55.693117 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf814156-7468-4cb7-8e50-5ba169ff98ff None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:55.697789 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:55.698379 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.698658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.698931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.699603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:55.701486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:55.775195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1353/2694] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:23:55 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:55.779645 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:55.790893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664730ec-1e33-4b57-af82-09a7dfb02b3b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:55.837255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:55.876563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:55.877428 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:55.877925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:55.878343 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:55.879399 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:55.881862 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:56.021602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1342/2695] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:23:55 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:23:56.024190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:56.034533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280153a-87f9-4716-953e-1bb8856304e0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:56.038977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:56.039602 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:56.039881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:56.040151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:56.040783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:56.042392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:56.053341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1354/2696] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:23:56.057319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:56.068210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3e38f-f60a-4485-9b95-d7160fe7f220 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:56.072969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6db834-8c23-4460-86ea-82ae7ca3a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:56.074104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db834-8c23-4460-86ea-82ae7ca3a7de admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:56.074104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db834-8c23-4460-86ea-82ae7ca3a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:56.074306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db834-8c23-4460-86ea-82ae7ca3a7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:56.074815 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6db834-8c23-4460-86ea-82ae7ca3a7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:56.076432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6db834-8c23-4460-86ea-82ae7ca3a7de admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:56.087197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1343/2697] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:23:56.091037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:56.101624 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e776f0-864a-4ce9-81ca-fa5d00c8bb70 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:56.106484 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:56.107137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:56.107456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:56.107733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] PATH_INFO: `/v3/projects/8879b3720c0d482d8118a0d2e5590b54/users/9a6e7e1e625247a3963aa8a0a29eb478/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:56.122898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8879b3720c0d482d8118a0d2e5590b54, user_id=9a6e7e1e625247a3963aa8a0a29eb478, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:56.125668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:56.190375 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1355/2698] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:23:56 2022] PUT /identity/v3/projects/8879b3720c0d482d8118a0d2e5590b54/users/9a6e7e1e625247a3963aa8a0a29eb478/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:23:56.195300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3e919-2abb-462e-b422-c4ee97112911 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:56.195613 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3e919-2abb-462e-b422-c4ee97112911 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:56.195930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3e919-2abb-462e-b422-c4ee97112911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:56.214768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0da3e919-2abb-462e-b422-c4ee97112911 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:56.216032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0da3e919-2abb-462e-b422-c4ee97112911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:56.275183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1344/2699] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:56.285711 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:56.287419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:56.331506 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:23:56.343247 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431f191-b49f-4933-95cf-6cb41f50a50f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:56.346836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:56.347469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:56.347749 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:56.348022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:56.356653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:56.358666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:56.367981 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1356/2700] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:57.644121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1ea9f-b623-4cfd-990e-ca8a45adac9c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:57.644665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1ea9f-b623-4cfd-990e-ca8a45adac9c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:57.644754 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1ea9f-b623-4cfd-990e-ca8a45adac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:57.677714 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-53b1ea9f-b623-4cfd-990e-ca8a45adac9c None None] MFA Rules not processed for user `9a6e7e1e625247a3963aa8a0a29eb478`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:57.678959 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-53b1ea9f-b623-4cfd-990e-ca8a45adac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:57.751071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1345/2701] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:23:57.762367 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:23:57.763925 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:57.825165 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad756b5-0b4a-4991-98f4-c5cda0d803b0 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:23:57.829383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:23:57.830026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:57.830308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:57.830583 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:57.839187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:23:57.840618 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:23:57.849614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1357/2702] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:23:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:23:59.560770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:23:59.561901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:23:59.561901 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:23:59.581791 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] MFA Rules not processed for user `9a6e7e1e625247a3963aa8a0a29eb478`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:23:59.583002 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:23:59.589868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1346/2703] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:23:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:02.596790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:02.616948 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b19351-ac42-4aed-80fe-afbb79693924 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:02.620644 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:02.621243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:02.621523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:02.621833 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:02.624601 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:02.685648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:02.687686 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:02.696921 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1358/2704] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:03.487842 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:03.489287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:03.555867 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:03.601144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ccd92-0eb0-4779-99b5-5cfc68016626 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:03.606673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:03.607330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:03.607624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:03.607922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:03.609672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:03.625749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:03.628190 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3423f0-f2ca-4244-b420-1ed37adeb1c9 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:03.645232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1347/2705] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:11.402823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:11.404278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:11.418081 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f548ef9-5418-457b-aef1-507651e91906 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:11.423727 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:11.424373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:11.424658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:11.425049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] PATH_INFO: `/v3/users/760dfa0e71e9476992153d71e8788a02` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:11.426299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=760dfa0e71e9476992153d71e8788a02)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:11.428313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:11.652665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760dfa0e71e9476992153d71e8788a02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:11.713849 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760dfa0e71e9476992153d71e8788a02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:11.714363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760dfa0e71e9476992153d71e8788a02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:11.715004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760dfa0e71e9476992153d71e8788a02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:11.724044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760dfa0e71e9476992153d71e8788a02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:11.732753 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:11.747684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1359/2706] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:11 2022] DELETE /identity/v3/users/760dfa0e71e9476992153d71e8788a02 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:11.749812 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:11.750603 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:11.750883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:11.773118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:11.773118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:11.839911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1348/2707] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:11.850040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:11.851502 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:11.853230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:11.854641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:11.942333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8eedc-9be7-4604-b031-8cb51feeec4a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:11.943109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca49da-77cc-4d2c-9030-1db536fe7903 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:11.946032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:11.946663 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:11.946937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:11.947038 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:11.947233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:11.947679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:11.947965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:11.948238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:11.949069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:11.958667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:11.960147 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:11.961762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:11.963152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:11.970649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1360/2708] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:11.974607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1349/2709] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:24:12.456231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:12.459719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:12.479954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1569d-af1a-49c1-857c-160155b0412d service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:12.485045 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:12.485359 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:12.485629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:12.486114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] PATH_INFO: `/v3/projects/75dba19cd411437a820b2b8c38b7437f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:12.487275 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75dba19cd411437a820b2b8c38b7437f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:12.489264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:12.602510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75dba19cd411437a820b2b8c38b7437f', 'request_id': 'req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:12.610738 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75dba19cd411437a820b2b8c38b7437f', 'request_id': 'req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:12.703676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75dba19cd411437a820b2b8c38b7437f', 'request_id': 'req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:12.716337 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] The token cache is being invalidate because project 75dba19cd411437a820b2b8c38b7437f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:12.716845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:12.717882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:12.720075 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1361/2710] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:12 2022] DELETE /identity/v3/projects/75dba19cd411437a820b2b8c38b7437f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:12.724240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:12.735082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6ed8-ddcd-42f2-b96d-2bb4c8415748 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:12.789580 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:12.803386 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:12.804039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:12.804321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:12.804599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] PATH_INFO: `/v3/users/c4a9f7408e3940a9a4ad0f89cc15fb9c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:12.805821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4a9f7408e3940a9a4ad0f89cc15fb9c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:12.807977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:12.995565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4a9f7408e3940a9a4ad0f89cc15fb9c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:13.059269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4a9f7408e3940a9a4ad0f89cc15fb9c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:13.060111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4a9f7408e3940a9a4ad0f89cc15fb9c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:13.060715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4a9f7408e3940a9a4ad0f89cc15fb9c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:13.070562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ce97e923-22c8-4623-a12b-eb9491185af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4a9f7408e3940a9a4ad0f89cc15fb9c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:13.089789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1350/2711] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:12 2022] DELETE /identity/v3/users/c4a9f7408e3940a9a4ad0f89cc15fb9c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:13.629187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:13.630709 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:13.645886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254f0a-d35e-4cf5-a7f4-d5e8e7492c02 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:13.720153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:13.720911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:13.721262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:13.721611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] PATH_INFO: `/v3/projects/34f215463b8847f1b4c865dc03f0c684` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:13.722873 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f215463b8847f1b4c865dc03f0c684)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:13.724927 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:13.734382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:13.907003 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f215463b8847f1b4c865dc03f0c684', 'request_id': 'req-34d49758-ff78-47fb-a21f-baf5fc910c04'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:13.914285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f215463b8847f1b4c865dc03f0c684', 'request_id': 'req-34d49758-ff78-47fb-a21f-baf5fc910c04'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:14.007566 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f215463b8847f1b4c865dc03f0c684', 'request_id': 'req-34d49758-ff78-47fb-a21f-baf5fc910c04'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:14.019536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] The token cache is being invalidate because project 34f215463b8847f1b4c865dc03f0c684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:14.019931 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:14.020639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-34d49758-ff78-47fb-a21f-baf5fc910c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:14.022142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1362/2712] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:13 2022] DELETE /identity/v3/projects/34f215463b8847f1b4c865dc03f0c684 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:14.027509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df236a9-4514-4634-8996-c515202e050a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.028053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df236a9-4514-4634-8996-c515202e050a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.028053 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df236a9-4514-4634-8996-c515202e050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.056396 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8df236a9-4514-4634-8996-c515202e050a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:14.057829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8df236a9-4514-4634-8996-c515202e050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:14.119201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1351/2713] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:14.127541 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.127859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.128136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.148428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:14.149843 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:14.158561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1363/2714] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:14.163006 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df236a9-4514-4634-8996-c515202e050a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:14.179834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df236a9-4514-4634-8996-c515202e050a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:14.184970 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:14.185803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.186872 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.187599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.188435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:14.188896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:14.190658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:14.206718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1352/2715] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:24:14.210244 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:14.229114 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35832383-d58e-4fc6-b019-f2617e0ced2f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:14.237495 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:14.238347 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.239021 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.239782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.241292 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:14.244211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:14.367665 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1364/2716] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:14 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:14.371733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:14.388890 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f7ad0-f086-464f-857b-53398b8011a9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:14.453509 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:14.463079 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:14.466443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.466443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.466443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.466443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:14.466873 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:14.567198 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1353/2717] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:14 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:14.568146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:14.579732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79364-8c04-487d-9bc1-c3a2fbbd635e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:14.584730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:14.585460 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.585812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.586155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.586876 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:14.588860 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:14.606225 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1365/2718] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:14.606911 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:14.616961 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631a36e-ccc5-43ac-9346-6bf87f200476 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:14.627672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77df3af-e86d-4caf-b4ca-3280a5f44973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:14.627672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df3af-e86d-4caf-b4ca-3280a5f44973 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.627672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df3af-e86d-4caf-b4ca-3280a5f44973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.627672 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df3af-e86d-4caf-b4ca-3280a5f44973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.628110 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77df3af-e86d-4caf-b4ca-3280a5f44973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:14.629071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77df3af-e86d-4caf-b4ca-3280a5f44973 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:14.641495 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1354/2719] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:24:14.645899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:14.658269 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201dfc-e42f-4640-af7e-1875ab09ff52 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:14.664093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:14.664954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.665401 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.665986 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] PATH_INFO: `/v3/projects/2e11fa99b40649c68550a1f6093cb5c8/users/db79b4525492497784c551b7ded53745/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.692416 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e11fa99b40649c68550a1f6093cb5c8, user_id=db79b4525492497784c551b7ded53745, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:14.695932 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:14.836887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1366/2720] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:14 2022] PUT /identity/v3/projects/2e11fa99b40649c68550a1f6093cb5c8/users/db79b4525492497784c551b7ded53745/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:14.849298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:14.849689 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:14.849977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:14.893864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] MFA Rules not processed for user `db79b4525492497784c551b7ded53745`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:14.895336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:15.011276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1355/2721] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:15.016511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:15.079836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2001cca-2bba-4fbe-9b03-463baa05975f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:15.084570 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:15.085198 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.085482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.085887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.086529 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:15.088468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:15.221127 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1367/2722] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:15 2022] POST /identity/v3/projects => generated 369 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:15.225336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:15.238488 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16a5fd-568e-4a15-a557-6799057adf55 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:15.301680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:15.302469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.302935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.303499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.304231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:15.306391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:15.438047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1356/2723] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:15 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:15.442384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:15.453736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fdd7c-1c82-4455-a158-28d61c56588e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:15.458517 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:15.459419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.464832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.464832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.464832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:15.464832 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:15.477957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1368/2724] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:15.479306 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:15.492504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34841330-7329-4543-a5e6-915e7e0cb986 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:15.500176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:15.500558 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.500916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.501298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] PATH_INFO: `/v3/projects/8eaca2ad205148b8a465517bb40f7232/users/52071c5e0b5d42ef8f37d5e43394a613/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.526773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eaca2ad205148b8a465517bb40f7232, user_id=52071c5e0b5d42ef8f37d5e43394a613, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:15.529905 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:15.635679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1357/2725] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:15 2022] PUT /identity/v3/projects/8eaca2ad205148b8a465517bb40f7232/users/52071c5e0b5d42ef8f37d5e43394a613/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:15.641312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:15.664965 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:15.669828 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327d89-df7d-407d-b4f9-03b89aa66e28 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:15.788252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cd6e1-bcc9-4b35-a1d4-6be68d72aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:15.788877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cd6e1-bcc9-4b35-a1d4-6be68d72aed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.789155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cd6e1-bcc9-4b35-a1d4-6be68d72aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.791328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cd6e1-bcc9-4b35-a1d4-6be68d72aed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.791328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cd6e1-bcc9-4b35-a1d4-6be68d72aed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:15.792404 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cd6e1-bcc9-4b35-a1d4-6be68d72aed2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:15.815350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1369/2726] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:15.822413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:15.822413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02116b4-0b38-4fc5-a8a6-c54586668e68 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:15.824767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:15.825419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.825703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.825977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] PATH_INFO: `/v3/projects/8eaca2ad205148b8a465517bb40f7232/users/52071c5e0b5d42ef8f37d5e43394a613/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.828531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eaca2ad205148b8a465517bb40f7232, user_id=52071c5e0b5d42ef8f37d5e43394a613, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:15.831174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:15.841231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1358/2727] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:15 2022] PUT /identity/v3/projects/8eaca2ad205148b8a465517bb40f7232/users/52071c5e0b5d42ef8f37d5e43394a613/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:15.852525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:15.852817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:15.853097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:15.881408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] MFA Rules not processed for user `52071c5e0b5d42ef8f37d5e43394a613`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:15.882629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:15.988689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1370/2728] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:16.002273 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:16.004499 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:16.024164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:16.096344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909ca75-5ac8-4b6a-846b-6db30d1fc964 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:16.102090 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:16.102482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:16.102771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:16.103077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:16.114804 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:16.116322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:16.128261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1359/2729] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:16.247102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:16.249041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:16.266436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872c05-f7b9-4de6-b77f-8b3757d0c68f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:16.271378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:16.272432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:16.272715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:16.272990 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:16.341243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:16.342895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:16.352276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1371/2730] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:20.700452 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:20.702039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:20.792596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c1a4-5268-4a90-b534-2356839d15c5 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:20.798023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:20.798780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:20.799144 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:20.799465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] PATH_INFO: `/v3/users/969b244842434bb09642122db0d159c0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:20.800740 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=969b244842434bb09642122db0d159c0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:20.802743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:21.026323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969b244842434bb09642122db0d159c0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.087218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969b244842434bb09642122db0d159c0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.088952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969b244842434bb09642122db0d159c0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.089894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969b244842434bb09642122db0d159c0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.099066 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969b244842434bb09642122db0d159c0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.119995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1360/2731] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:20 2022] DELETE /identity/v3/users/969b244842434bb09642122db0d159c0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:21.452934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:21.468611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec98a4-902b-439a-a7e6-532980c38b02 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:21.497168 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:21.535232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:21.535779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:21.536207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:21.536489 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] PATH_INFO: `/v3/projects/9ef16a5156d9421e85b4106cce719b4f` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:21.537609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef16a5156d9421e85b4106cce719b4f)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:21.539581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:21.647730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef16a5156d9421e85b4106cce719b4f', 'request_id': 'req-76717e6f-e821-4bb4-b572-d02331e96018'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.654299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef16a5156d9421e85b4106cce719b4f', 'request_id': 'req-76717e6f-e821-4bb4-b572-d02331e96018'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.715976 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef16a5156d9421e85b4106cce719b4f', 'request_id': 'req-76717e6f-e821-4bb4-b572-d02331e96018'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.728994 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] The token cache is being invalidate because project 9ef16a5156d9421e85b4106cce719b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:21.730019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.730804 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:21.731436 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:21.732524 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1372/2732] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:21 2022] DELETE /identity/v3/projects/9ef16a5156d9421e85b4106cce719b4f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:21.733334 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:21.740742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:21.742142 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:21.748514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04624cee-cd59-41ee-a7ab-02fa46f3f390 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:21.760351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:21.818298 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:21.818451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76717e6f-e821-4bb4-b572-d02331e96018 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:21.818940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:21.819265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:21.819566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] PATH_INFO: `/v3/users/db79b4525492497784c551b7ded53745` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:21.820747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=db79b4525492497784c551b7ded53745)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:21.822637 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:21.823639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:21.824493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:21.824779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:21.825112 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] PATH_INFO: `/v3/users/82daac0f2c764609810cbd188b653ccb` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:21.826389 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82daac0f2c764609810cbd188b653ccb)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:21.828755 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:21.983512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db79b4525492497784c551b7ded53745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.014058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82daac0f2c764609810cbd188b653ccb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.092898 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db79b4525492497784c551b7ded53745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.093542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db79b4525492497784c551b7ded53745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.094162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db79b4525492497784c551b7ded53745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.101792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3142e2ad-c24e-46bc-8550-2b3e776f6eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db79b4525492497784c551b7ded53745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.121254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1361/2733] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:21 2022] DELETE /identity/v3/users/db79b4525492497784c551b7ded53745 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:22.126357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:22.126644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:22.129044 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:22.146072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:22.147388 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:22.148700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82daac0f2c764609810cbd188b653ccb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.149448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82daac0f2c764609810cbd188b653ccb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.150163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82daac0f2c764609810cbd188b653ccb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.153356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:22.159037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82daac0f2c764609810cbd188b653ccb'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.182387 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1373/2734] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:21 2022] DELETE /identity/v3/users/82daac0f2c764609810cbd188b653ccb => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:22.214688 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1362/2735] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:22.224845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:22.226266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:22.288895 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49149686-3060-4d1c-b2be-6416bfda406a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:22.294451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:22.295879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:22.296385 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:22.296669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:22.386710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:22.392838 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:22.399097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1374/2736] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:22.516425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:22.524039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:22.535683 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10893bb-5794-4c30-899a-5d786ddbe6f5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:22.540600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:22.541760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:22.542194 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:22.542470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] PATH_INFO: `/v3/projects/231c305c10ea46008717a137a8543f89` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:22.543880 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231c305c10ea46008717a137a8543f89)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:22.545769 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:22.681681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:22.700839 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231c305c10ea46008717a137a8543f89', 'request_id': 'req-9d64f9d4-01db-4822-bfed-874f567f7d42'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.707725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231c305c10ea46008717a137a8543f89', 'request_id': 'req-9d64f9d4-01db-4822-bfed-874f567f7d42'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.770922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231c305c10ea46008717a137a8543f89', 'request_id': 'req-9d64f9d4-01db-4822-bfed-874f567f7d42'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.783377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] The token cache is being invalidate because project 231c305c10ea46008717a137a8543f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:22.783673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.784317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:22.785818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1363/2737] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:22 2022] DELETE /identity/v3/projects/231c305c10ea46008717a137a8543f89 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:22.871151 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:22.873887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:22.892299 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635579-1b1f-4a4f-b795-fd3d2269d4a2 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:22.963382 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:22.964051 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:22.964336 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:22.964614 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] PATH_INFO: `/v3/projects/2e11fa99b40649c68550a1f6093cb5c8` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:22.965959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e11fa99b40649c68550a1f6093cb5c8)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:22.967885 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:23.066501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:23.083617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e11fa99b40649c68550a1f6093cb5c8', 'request_id': 'req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.089813 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e11fa99b40649c68550a1f6093cb5c8', 'request_id': 'req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.144568 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e11fa99b40649c68550a1f6093cb5c8', 'request_id': 'req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.157411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] The token cache is being invalidate because project 2e11fa99b40649c68550a1f6093cb5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:23.157713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.158505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.160177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1375/2738] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:22 2022] DELETE /identity/v3/projects/2e11fa99b40649c68550a1f6093cb5c8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:23.164319 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:23.165784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:23.179962 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64f9d4-01db-4822-bfed-874f567f7d42 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:23.240625 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:23.244691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:23.244691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:23.244691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] PATH_INFO: `/v3/users/52071c5e0b5d42ef8f37d5e43394a613` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:23.244691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52071c5e0b5d42ef8f37d5e43394a613)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:23.245086 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:23.490466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52071c5e0b5d42ef8f37d5e43394a613'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.549910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52071c5e0b5d42ef8f37d5e43394a613'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.550522 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52071c5e0b5d42ef8f37d5e43394a613'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.551124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52071c5e0b5d42ef8f37d5e43394a613'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.559182 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7b165fe0-af69-4aa3-be5b-84e676ee388a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52071c5e0b5d42ef8f37d5e43394a613'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:23.578299 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1364/2739] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:23 2022] DELETE /identity/v3/users/52071c5e0b5d42ef8f37d5e43394a613 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:24.024690 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.026959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:24.045262 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c71ea-04c8-4774-9bd9-c7dc04cebb0b admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.109470 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.110130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.110411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.110781 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] PATH_INFO: `/v3/projects/8eaca2ad205148b8a465517bb40f7232` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.112026 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eaca2ad205148b8a465517bb40f7232)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:24.114137 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:24.217949 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eaca2ad205148b8a465517bb40f7232', 'request_id': 'req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:24.224821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eaca2ad205148b8a465517bb40f7232', 'request_id': 'req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:24.378486 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eaca2ad205148b8a465517bb40f7232', 'request_id': 'req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:24.397455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] The token cache is being invalidate because project 8eaca2ad205148b8a465517bb40f7232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:24.416013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:24.416013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-89fb9b30-6fd1-4e8e-a7e3-9b0c7c5293e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:24.416013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1376/2740] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:24 2022] DELETE /identity/v3/projects/8eaca2ad205148b8a465517bb40f7232 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:24.416752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.416752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.416752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.436667 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:24.437405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:24.481095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:24.510291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1365/2741] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:24.518853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.519139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.519437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.539920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:24.541266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:24.549533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1377/2742] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:24.554380 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.566466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3c301-bf46-43fc-8739-1379317e449f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.571426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.571992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.572274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.572549 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.573203 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:24.573516 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:24.575139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:24.588142 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1366/2743] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:24:24.592799 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.602303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c95d8-92a1-450f-a77f-c285f2db51f9 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.607093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.607775 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.608056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.608331 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.609007 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:24.610906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:24.737900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1378/2744] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:24 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:24.739560 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.750527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d91b42-cb37-4de4-a9e9-19e26b6be0b2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.809909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.810568 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.810856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.811134 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.811818 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:24.814327 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:24.895889 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1367/2745] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:24 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:24.897818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.909579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c1656-8ace-4bae-baf8-a999fb105162 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.914997 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.915898 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.916204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.916505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.917201 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:24.918786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:24.929770 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1379/2746] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:24.935256 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.946344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28ce5-9ef5-4643-a0e7-afefedb8a790 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.951241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bab0f-5063-4326-b0aa-b28ba433b434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.951849 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bab0f-5063-4326-b0aa-b28ba433b434 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.952129 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bab0f-5063-4326-b0aa-b28ba433b434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.952403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bab0f-5063-4326-b0aa-b28ba433b434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:24.953080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bab0f-5063-4326-b0aa-b28ba433b434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:24.954664 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bab0f-5063-4326-b0aa-b28ba433b434 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:24.965377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1368/2747] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:24.970414 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:24.977022 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:24.981097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de144c-39ce-4055-b024-211ea3abcfe7 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:24.986886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:24.992657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:24.992657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:24.992657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] PATH_INFO: `/v3/projects/1572eb2ef4cb4415aa5451a08e593b52/users/3ff221165d2d4e219fbb96b8c096e6bc/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.003648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1572eb2ef4cb4415aa5451a08e593b52, user_id=3ff221165d2d4e219fbb96b8c096e6bc, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.006538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.087790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1380/2748] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:24 2022] PUT /identity/v3/projects/1572eb2ef4cb4415aa5451a08e593b52/users/3ff221165d2d4e219fbb96b8c096e6bc/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:25.098937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.104979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.104979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.129550 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] MFA Rules not processed for user `3ff221165d2d4e219fbb96b8c096e6bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:25.130853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:25.198009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1369/2749] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:25.202503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.265765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf237-c6f9-40ac-a3c9-edb8a1fed664 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:25.270369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:25.270974 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.271285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.271556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.272185 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.274102 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.412415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1381/2750] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:25 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:25.416795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.427893 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ed1f3-a253-4d79-a642-eb913b25c677 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:25.459395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:25.485344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:25.485995 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.486441 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.487104 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.487814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.489736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.590604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1370/2751] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:25 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:25.594723 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.605113 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991ec4e-98f2-42e6-be15-051f37caf929 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:25.610276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:25.610869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.611153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.611668 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.616306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.616306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.624261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1382/2752] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:25.628971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.639021 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63b453-edaf-4e9c-a7f2-194ed23534da admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:25.644260 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:25.644939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.645222 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.645527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] PATH_INFO: `/v3/projects/5747730dcb9b4593985d9a33db2f4ba3/users/2af2811a881e485694ddc8b73af486e4/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.661156 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5747730dcb9b4593985d9a33db2f4ba3, user_id=2af2811a881e485694ddc8b73af486e4, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.663855 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.727801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1371/2753] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:25 2022] PUT /identity/v3/projects/5747730dcb9b4593985d9a33db2f4ba3/users/2af2811a881e485694ddc8b73af486e4/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:25.731788 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.742815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777389f-a905-4b8d-bc96-4804e016ab42 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:25.808397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a4b33-6d53-4dc0-bfd8-bd50d575567d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:25.811973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4b33-6d53-4dc0-bfd8-bd50d575567d admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.811973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4b33-6d53-4dc0-bfd8-bd50d575567d admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.811973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4b33-6d53-4dc0-bfd8-bd50d575567d admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.811973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a4b33-6d53-4dc0-bfd8-bd50d575567d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.811973 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a4b33-6d53-4dc0-bfd8-bd50d575567d admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.823536 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1383/2754] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:25.827677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.837974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039f95a-42d0-4e32-a03d-823c79d023e1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:25.842940 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:25.843624 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.843912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.844188 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] PATH_INFO: `/v3/projects/5747730dcb9b4593985d9a33db2f4ba3/users/2af2811a881e485694ddc8b73af486e4/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.846276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5747730dcb9b4593985d9a33db2f4ba3, user_id=2af2811a881e485694ddc8b73af486e4, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:25.849510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:25.861174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1372/2755] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:25 2022] PUT /identity/v3/projects/5747730dcb9b4593985d9a33db2f4ba3/users/2af2811a881e485694ddc8b73af486e4/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:25.871574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:25.871863 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:25.872163 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:25.901371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] MFA Rules not processed for user `2af2811a881e485694ddc8b73af486e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:25.902594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:25.943715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:25.973210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1384/2756] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:25.986811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:25.988591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:26.050454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cc6ce-87ea-43cf-ae38-73c67eb93d9f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:26.054286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:26.054896 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:26.055241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:26.055491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:26.065206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:26.066644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:26.076247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1373/2757] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:26.974371 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:26.983494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a7ce9-1ace-4df6-ab22-23a4468b426f None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:26.988667 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af2811a881e485694ddc8b73af486e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5747730dcb9b4593985d9a33db2f4ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:26.989278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:26.989567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:26.989845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:26.990498 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:26.990810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:26.992485 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:27.005325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1385/2758] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:24:27.009419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:27.018542 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d25d0f-24b2-4ae2-a8d3-a49c58df254b service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:27.023963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af2811a881e485694ddc8b73af486e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5747730dcb9b4593985d9a33db2f4ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:27.024571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:27.024857 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:27.025136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:27.025778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:27.028209 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:27.100974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1374/2759] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:27 2022] POST /identity/v3/projects => generated 355 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:27.105096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:27.115714 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1986b0-b56a-412f-813a-35950911c42a tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:27.172068 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af2811a881e485694ddc8b73af486e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5747730dcb9b4593985d9a33db2f4ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:27.172684 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:27.172971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:27.173251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:27.173892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:27.174200 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:27.175829 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:27.188666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1386/2760] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:24:27.231059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:27.242120 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3ef34-c2fe-429d-8fdb-0cbd57a2a7cc tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:27.247082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af2811a881e485694ddc8b73af486e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5747730dcb9b4593985d9a33db2f4ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:27.247767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:27.248049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:27.248321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] PATH_INFO: `/v3/projects/668cd1e3781e4f35a3c6d946f43e16aa` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:27.249528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorizing `identity:delete_project(project_id=668cd1e3781e4f35a3c6d946f43e16aa)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:27.251468 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:27.334532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668cd1e3781e4f35a3c6d946f43e16aa', 'request_id': 'req-86181d69-0968-4bc5-b696-28a095192cfe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:27.341403 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668cd1e3781e4f35a3c6d946f43e16aa', 'request_id': 'req-86181d69-0968-4bc5-b696-28a095192cfe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:27.411390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668cd1e3781e4f35a3c6d946f43e16aa', 'request_id': 'req-86181d69-0968-4bc5-b696-28a095192cfe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:27.412879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:27.422927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] The token cache is being invalidate because project 668cd1e3781e4f35a3c6d946f43e16aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:27.423259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:27.423963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:27.425504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1375/2761] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:27 2022] DELETE /identity/v3/projects/668cd1e3781e4f35a3c6d946f43e16aa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:27.727101 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:27.728998 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:27.791899 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd38124-3546-42c3-af03-384c767d9ad5 tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:27.846544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:27.847195 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:27.847482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:27.847791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:27.850363 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:27.881700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:27.914917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:27.916426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21720cbc-13cc-48a6-8a25-6479cec93b37 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:27.926478 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1387/2762] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:28.000445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:28.001978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:28.069100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86181d69-0968-4bc5-b696-28a095192cfe tempest-VolumeQuotaClassesTest-965358106 tempest-VolumeQuotaClassesTest-965358106-project-admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:28.074047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:28.074701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:28.074986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:28.075295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] PATH_INFO: `/v3/users/3ff221165d2d4e219fbb96b8c096e6bc` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:28.076490 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff221165d2d4e219fbb96b8c096e6bc)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:28.078448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:28.296174 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff221165d2d4e219fbb96b8c096e6bc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:28.433702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff221165d2d4e219fbb96b8c096e6bc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:28.434419 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff221165d2d4e219fbb96b8c096e6bc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:28.435049 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff221165d2d4e219fbb96b8c096e6bc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:28.446121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff221165d2d4e219fbb96b8c096e6bc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:28.468697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1376/2763] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:27 2022] DELETE /identity/v3/users/3ff221165d2d4e219fbb96b8c096e6bc => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:28.473406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:28.474434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:28.474636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:28.494730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:28.495978 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:28.560380 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1388/2764] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:28.570385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:28.571831 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:28.628781 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0a7f4-d6af-4254-95b3-b8a9d380f51a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:28.632392 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:28.632993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:28.633270 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:28.633539 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:28.642293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:28.643748 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:28.654024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1377/2765] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:29.066166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:29.069825 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:29.086203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f7c8f-2a8e-4f14-b96d-456b0c350960 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:29.091967 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:29.092595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:29.092918 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:29.093130 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] PATH_INFO: `/v3/projects/1572eb2ef4cb4415aa5451a08e593b52` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:29.094322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1572eb2ef4cb4415aa5451a08e593b52)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:29.096189 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:29.198187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1572eb2ef4cb4415aa5451a08e593b52', 'request_id': 'req-4fe0a40c-03a4-4805-bda0-24c2f3977004'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.204174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1572eb2ef4cb4415aa5451a08e593b52', 'request_id': 'req-4fe0a40c-03a4-4805-bda0-24c2f3977004'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.281922 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1572eb2ef4cb4415aa5451a08e593b52', 'request_id': 'req-4fe0a40c-03a4-4805-bda0-24c2f3977004'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.293777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] The token cache is being invalidate because project 1572eb2ef4cb4415aa5451a08e593b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:29.294065 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.294713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.296693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1389/2766] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:29 2022] DELETE /identity/v3/projects/1572eb2ef4cb4415aa5451a08e593b52 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:29.300790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:29.311463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95eeb21-5693-414e-8660-a8b76f3b32cf service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:29.319870 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:29.369925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:29.370614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:29.372473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:29.372749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] PATH_INFO: `/v3/users/2af2811a881e485694ddc8b73af486e4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:29.374011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af2811a881e485694ddc8b73af486e4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:29.376011 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:29.564910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af2811a881e485694ddc8b73af486e4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.616448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af2811a881e485694ddc8b73af486e4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.617070 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af2811a881e485694ddc8b73af486e4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.617692 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af2811a881e485694ddc8b73af486e4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.624976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89bacdd-ea31-469c-b78f-f36d17679258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af2811a881e485694ddc8b73af486e4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:29.642215 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1378/2767] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:29 2022] DELETE /identity/v3/users/2af2811a881e485694ddc8b73af486e4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:30.097875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.099936 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:30.115481 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0a40c-03a4-4805-bda0-24c2f3977004 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.169008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:30.178671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.179364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.179672 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.179946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] PATH_INFO: `/v3/projects/5747730dcb9b4593985d9a33db2f4ba3` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.181218 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5747730dcb9b4593985d9a33db2f4ba3)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.183181 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:30.346756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5747730dcb9b4593985d9a33db2f4ba3', 'request_id': 'req-1749ec53-9017-495b-874d-90e13a1180d3'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:30.353527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5747730dcb9b4593985d9a33db2f4ba3', 'request_id': 'req-1749ec53-9017-495b-874d-90e13a1180d3'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:30.418348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5747730dcb9b4593985d9a33db2f4ba3', 'request_id': 'req-1749ec53-9017-495b-874d-90e13a1180d3'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:30.430479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] The token cache is being invalidate because project 5747730dcb9b4593985d9a33db2f4ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:30.430779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:30.431461 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-1749ec53-9017-495b-874d-90e13a1180d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:30.433266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1390/2768] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:30 2022] DELETE /identity/v3/projects/5747730dcb9b4593985d9a33db2f4ba3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:30.438112 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.438623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.438778 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.458548 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:30.459775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:30.533992 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1379/2769] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:30.542167 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.542457 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.542774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.563135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:30.564374 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:30.572312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1391/2770] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:30.576360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.585909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d9c09-a2b7-4b24-a27e-49f712e77656 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.590599 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.591231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.591543 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.591821 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.592492 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:30.592801 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.594454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:30.606978 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1380/2771] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:24:30.611710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.621499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e63b-5179-4ec2-92c2-b719d5eaeb39 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.626432 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.627029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.627366 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.627643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.628315 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.630680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:30.699479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1392/2772] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:30 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:30.703677 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.713743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd57e0-fd75-4ecb-9189-83c76396f209 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.727364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:30.773402 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.774095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.774538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.774820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.775680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.777882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:30.875553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1381/2773] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:30 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:30.879574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.889855 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f698cd-5d0f-4d6f-9faf-8a947009b71a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.894740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.895378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.896143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.896451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.897095 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.898760 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:30.909232 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1393/2774] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:30.913658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.923498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297c1f-a107-45aa-9d38-cbfc03dc879d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.928208 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481e9e8-2baf-4c40-a9e5-21b94f5aeed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.931287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481e9e8-2baf-4c40-a9e5-21b94f5aeed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.931287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481e9e8-2baf-4c40-a9e5-21b94f5aeed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.931287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481e9e8-2baf-4c40-a9e5-21b94f5aeed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.931287 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481e9e8-2baf-4c40-a9e5-21b94f5aeed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.931703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481e9e8-2baf-4c40-a9e5-21b94f5aeed5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:30.941181 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1382/2775] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:30.945196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:30.955072 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422abcae-e0ef-47e2-87ab-fc7e0630a01f admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:30.960627 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:30.961279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:30.961559 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:30.961834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] PATH_INFO: `/v3/projects/2e1e32411d3d45e0addd245a2f227bd3/users/e71f5a790b61471198995fd2703533d1/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:30.977533 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1e32411d3d45e0addd245a2f227bd3, user_id=e71f5a790b61471198995fd2703533d1, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:30.980396 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.069942 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1394/2776] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:30 2022] PUT /identity/v3/projects/2e1e32411d3d45e0addd245a2f227bd3/users/e71f5a790b61471198995fd2703533d1/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:31.081150 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.081435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.081710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.109718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] MFA Rules not processed for user `e71f5a790b61471198995fd2703533d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:31.110930 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:31.174974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1383/2777] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:31.179356 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:31.241040 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b177cc-b2c8-4179-84c8-d432304b43f3 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:31.247012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:31.247683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.247935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.248210 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.248859 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:31.250773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.348868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1395/2778] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:31 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:31.352977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:31.363591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5674d5f-e7c4-499e-b98c-590f9fa6a165 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:31.421582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:31.422214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.422481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.422757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.423589 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:31.425760 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.521325 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1384/2779] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:31 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:31.525097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:31.531503 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:31.536820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31a2bf-ae33-4ada-accc-a7428cfe7824 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:31.542724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:31.542724 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.548538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.548815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.549487 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:31.551077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.562608 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1396/2780] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:31.566579 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:31.577564 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12258c7-c038-437c-a1ff-e9860a3f4dc0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:31.582202 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3bff9-dee1-4e4a-814e-684624cd71fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:31.582841 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bff9-dee1-4e4a-814e-684624cd71fb admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.583124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bff9-dee1-4e4a-814e-684624cd71fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.583453 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bff9-dee1-4e4a-814e-684624cd71fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.584075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3bff9-dee1-4e4a-814e-684624cd71fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:31.585668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3bff9-dee1-4e4a-814e-684624cd71fb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.595553 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1385/2781] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:31.599523 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:31.609383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea301d-e5ab-44bd-8178-935421b8b2ee admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:31.615109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:31.615780 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.616054 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.616321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] PATH_INFO: `/v3/projects/08e8f31934ea4bd8ba753eedff9505a5/users/4287a5b3516942588dbeed58215166c2/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.631082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e8f31934ea4bd8ba753eedff9505a5, user_id=4287a5b3516942588dbeed58215166c2, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:31.633720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.691971 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1397/2782] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:31 2022] PUT /identity/v3/projects/08e8f31934ea4bd8ba753eedff9505a5/users/4287a5b3516942588dbeed58215166c2/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:31.702897 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.703231 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.703511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.711687 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:31.731124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] MFA Rules not processed for user `4287a5b3516942588dbeed58215166c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:31.732415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:31.796868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1386/2783] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:31.801464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:31.868679 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436951cc-c814-492e-8154-24ef94058092 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:31.873451 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:31.874106 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:31.874383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:31.874653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:31.875350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:31.877273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:31.983790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1398/2784] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:31 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:31.988311 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:32.000514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e40fe-453b-48e4-b4da-f0f32b74cae4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:32.057147 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:32.057747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.058029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.058296 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.058931 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:32.060864 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:32.137111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1387/2785] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:31 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:32.141202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:32.152455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c759561-06ae-47e1-8f39-fbff98b06fd2 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:32.157490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:32.158129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.159253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.159253 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.159513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:32.161184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:32.173094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1399/2786] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:32.178069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:32.188368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb83ce9-0f8a-4550-ba00-3af11a85e45f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:32.193080 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:32.193777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.194069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.194351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] PATH_INFO: `/v3/projects/91ac53e0baa8457298bf54bd81c6f93f/users/c927ebd067684ada8f4631f42def60b0/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.210075 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ac53e0baa8457298bf54bd81c6f93f, user_id=c927ebd067684ada8f4631f42def60b0, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:32.212792 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:32.328575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1388/2787] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:32 2022] PUT /identity/v3/projects/91ac53e0baa8457298bf54bd81c6f93f/users/c927ebd067684ada8f4631f42def60b0/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:32.332611 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:32.343748 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3475657-7e80-4881-b5a9-26982f4ba3b8 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:32.406772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eda17-9d89-43e5-9b82-745232b8d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:32.407411 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eda17-9d89-43e5-9b82-745232b8d024 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.407700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eda17-9d89-43e5-9b82-745232b8d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.407977 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eda17-9d89-43e5-9b82-745232b8d024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.408629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eda17-9d89-43e5-9b82-745232b8d024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:32.413226 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eda17-9d89-43e5-9b82-745232b8d024 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:32.420671 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1400/2788] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:32.425153 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:32.436462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d1a20-2646-4696-8023-5767debbcb60 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:32.441405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:32.442083 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.442367 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.443192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] PATH_INFO: `/v3/projects/91ac53e0baa8457298bf54bd81c6f93f/users/c927ebd067684ada8f4631f42def60b0/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.445158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ac53e0baa8457298bf54bd81c6f93f, user_id=c927ebd067684ada8f4631f42def60b0, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:32.447845 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:32.458063 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1389/2789] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:32 2022] PUT /identity/v3/projects/91ac53e0baa8457298bf54bd81c6f93f/users/c927ebd067684ada8f4631f42def60b0/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:32.469378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.469698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.469983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.498830 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] MFA Rules not processed for user `c927ebd067684ada8f4631f42def60b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:32.500082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:32.528187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:32.579053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1401/2790] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:32.587653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:32.589355 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:32.649430 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:32.654947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a118d6-2f9c-41ae-b73b-0e741f644277 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:32.658827 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e05e43-d08e-4078-a126-8a348b839b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:32.659669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e05e43-d08e-4078-a126-8a348b839b1e service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:32.659955 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e05e43-d08e-4078-a126-8a348b839b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:32.660232 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e05e43-d08e-4078-a126-8a348b839b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:32.670461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e05e43-d08e-4078-a126-8a348b839b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:32.671928 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e05e43-d08e-4078-a126-8a348b839b1e service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:32.682685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1390/2791] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:33.975029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:33.985212 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bd621-61bc-4166-92aa-05c49ca8f3c3 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:33.988821 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:33.989933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:33.990213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:33.990479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:34.049640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:34.051279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:34.061428 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1402/2792] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:36.484965 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:36.485470 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:36.485571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:36.504662 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] MFA Rules not processed for user `7ddb2572cac446aa92a0bcf170d9e8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:36.506481 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:36.560999 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1391/2793] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:40.019651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:40.019651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:40.030433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3526b-00cc-4de3-ba25-2f8194cbe4dd service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:40.034900 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12569f8-6238-4ff8-8c26-06e6a0816d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:40.035615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12569f8-6238-4ff8-8c26-06e6a0816d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:40.035955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12569f8-6238-4ff8-8c26-06e6a0816d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:40.036297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12569f8-6238-4ff8-8c26-06e6a0816d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:40.097842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12569f8-6238-4ff8-8c26-06e6a0816d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:40.099870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12569f8-6238-4ff8-8c26-06e6a0816d15 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:40.110089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1403/2794] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:46.553957 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=30 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:47.739803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:47.817971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a2a04-b9e1-4df5-a673-be0374d21b63 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:47.822909 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:47.827200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:47.827200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:47.827200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] PATH_INFO: `/v3/users/e71f5a790b61471198995fd2703533d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:47.827200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71f5a790b61471198995fd2703533d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:47.827606 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:48.008971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71f5a790b61471198995fd2703533d1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.096102 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71f5a790b61471198995fd2703533d1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.096741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71f5a790b61471198995fd2703533d1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.097357 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71f5a790b61471198995fd2703533d1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.105466 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71f5a790b61471198995fd2703533d1'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.122617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1392/2795] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:47 2022] DELETE /identity/v3/users/e71f5a790b61471198995fd2703533d1 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:48.134435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:48.134917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:48.135030 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:48.154768 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:48.156041 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:48.206693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=100 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:48.215590 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1404/2796] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:48.226840 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:48.228498 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:48.288159 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2213d-82f2-4b5a-852b-c9c723bd8bcc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:48.291789 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:48.292373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:48.292655 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:48.292927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:48.301974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:48.303508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:48.312941 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1393/2797] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:48.793879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:48.793879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:48.809376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccb417-603e-4338-b0eb-42489202b969 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:48.816220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:48.817488 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:48.817774 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:48.818049 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] PATH_INFO: `/v3/projects/2e1e32411d3d45e0addd245a2f227bd3` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:48.819384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1e32411d3d45e0addd245a2f227bd3)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:48.821554 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:48.927878 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1e32411d3d45e0addd245a2f227bd3', 'request_id': 'req-90544311-3e3b-44a0-a492-d9e720a7e7a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.934174 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1e32411d3d45e0addd245a2f227bd3', 'request_id': 'req-90544311-3e3b-44a0-a492-d9e720a7e7a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.987052 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1e32411d3d45e0addd245a2f227bd3', 'request_id': 'req-90544311-3e3b-44a0-a492-d9e720a7e7a4'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.998311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] The token cache is being invalidate because project 2e1e32411d3d45e0addd245a2f227bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:48.998603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:48.999266 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:49.000773 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1405/2798] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:48 2022] DELETE /identity/v3/projects/2e1e32411d3d45e0addd245a2f227bd3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:49.005484 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:49.015346 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ec43e-2bb4-4ab3-b8af-a2151625f427 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:49.033443 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:49.073617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:49.074494 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:49.074925 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:49.075289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] PATH_INFO: `/v3/users/4287a5b3516942588dbeed58215166c2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:49.076586 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4287a5b3516942588dbeed58215166c2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:49.078511 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:49.321094 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4287a5b3516942588dbeed58215166c2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:49.415595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4287a5b3516942588dbeed58215166c2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:49.416234 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4287a5b3516942588dbeed58215166c2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:49.419685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4287a5b3516942588dbeed58215166c2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:49.425230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4287a5b3516942588dbeed58215166c2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:49.443745 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1394/2799] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:49 2022] DELETE /identity/v3/users/4287a5b3516942588dbeed58215166c2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:49.797370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:49.798906 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:49.812456 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90544311-3e3b-44a0-a492-d9e720a7e7a4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:49.872032 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:49.872676 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:49.872962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:49.873240 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] PATH_INFO: `/v3/users/3fed882c541b47fb847bd28848db82fc` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:49.874400 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fed882c541b47fb847bd28848db82fc)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:49.876360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:49.920685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:49.922291 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:49.939310 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0053cff-5c17-4e64-80c5-4922672a6c8a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:49.945752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:49.946457 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:49.946749 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:49.947023 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] PATH_INFO: `/v3/projects/08e8f31934ea4bd8ba753eedff9505a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:49.948320 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e8f31934ea4bd8ba753eedff9505a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:49.950247 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:50.087627 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e8f31934ea4bd8ba753eedff9505a5', 'request_id': 'req-a4247b4b-6a04-4fc2-8b05-60197e0eead5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.088297 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fed882c541b47fb847bd28848db82fc'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.094753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e8f31934ea4bd8ba753eedff9505a5', 'request_id': 'req-a4247b4b-6a04-4fc2-8b05-60197e0eead5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.177779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fed882c541b47fb847bd28848db82fc'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.178325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fed882c541b47fb847bd28848db82fc'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.178933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fed882c541b47fb847bd28848db82fc'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.186431 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ab1ea466-ba0f-4ac6-8d39-6a8f5c839322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fed882c541b47fb847bd28848db82fc'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.204617 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1406/2800] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:49 2022] DELETE /identity/v3/users/3fed882c541b47fb847bd28848db82fc => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:50.209640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:50.209933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:50.210211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:50.217135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:50.229643 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:50.230858 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:50.241372 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e8f31934ea4bd8ba753eedff9505a5', 'request_id': 'req-a4247b4b-6a04-4fc2-8b05-60197e0eead5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.255032 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] The token cache is being invalidate because project 08e8f31934ea4bd8ba753eedff9505a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:50.255653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.256770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.258770 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1395/2801] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:49 2022] DELETE /identity/v3/projects/08e8f31934ea4bd8ba753eedff9505a5 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:50.263508 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:50.265108 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:50.297947 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:50.302328 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1407/2802] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:50.312140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:50.313666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:50.336555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247b4b-6a04-4fc2-8b05-60197e0eead5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:50.341295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:50.341927 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:50.342224 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:50.342489 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] PATH_INFO: `/v3/users/c927ebd067684ada8f4631f42def60b0` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:50.343665 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c927ebd067684ada8f4631f42def60b0)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:50.345618 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:50.372633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78428c-ef2b-4967-93bb-cab3b96fa25b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:50.376306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:50.377006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:50.377285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:50.377556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:50.387896 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:50.389357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:50.399392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1408/2803] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:24:50.540405 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c927ebd067684ada8f4631f42def60b0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.612100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c927ebd067684ada8f4631f42def60b0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.612744 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c927ebd067684ada8f4631f42def60b0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.613421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c927ebd067684ada8f4631f42def60b0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.622439 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c927ebd067684ada8f4631f42def60b0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:50.640569 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1396/2804] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:50 2022] DELETE /identity/v3/users/c927ebd067684ada8f4631f42def60b0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:50.781214 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:50.783103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:50.806710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7cfd1-ec53-4a57-8fc2-621442878a79 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:50.881278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:50.882067 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:50.882628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:50.883008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] PATH_INFO: `/v3/projects/646eb6c170834f0ab978bd07582277ca` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:50.884412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] RBAC: Authorizing `identity:delete_project(project_id=646eb6c170834f0ab978bd07582277ca)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:50.886360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.010252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:51.012333 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646eb6c170834f0ab978bd07582277ca', 'request_id': 'req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.027062 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646eb6c170834f0ab978bd07582277ca', 'request_id': 'req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.101567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646eb6c170834f0ab978bd07582277ca', 'request_id': 'req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.108537 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.111448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:51.113826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] The token cache is being invalidate because project 646eb6c170834f0ab978bd07582277ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:51.114197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.115012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.116875 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1409/2805] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:50 2022] DELETE /identity/v3/projects/646eb6c170834f0ab978bd07582277ca => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:51.120929 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.122316 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:51.124097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92da084-ac49-4c84-aac9-fb25e6ddacd7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:51.183449 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:51.184255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.184538 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.184814 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] PATH_INFO: `/v3/projects/91ac53e0baa8457298bf54bd81c6f93f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.185979 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ac53e0baa8457298bf54bd81c6f93f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:51.187924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.193629 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25f9d9-1d6b-4dc8-a0e8-fc896c40601c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:51.198448 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:51.199099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.199728 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.200000 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] PATH_INFO: `/v3/users/6ffa47f98c5040c9a6805c3fdbbc9f91` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.201135 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffa47f98c5040c9a6805c3fdbbc9f91)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:51.203027 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.313438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:51.316288 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ac53e0baa8457298bf54bd81c6f93f', 'request_id': 'req-5bdd8db0-31de-4740-b282-95a9ec28aeff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.323216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ac53e0baa8457298bf54bd81c6f93f', 'request_id': 'req-5bdd8db0-31de-4740-b282-95a9ec28aeff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.398803 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ac53e0baa8457298bf54bd81c6f93f', 'request_id': 'req-5bdd8db0-31de-4740-b282-95a9ec28aeff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.410062 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] The token cache is being invalidate because project 91ac53e0baa8457298bf54bd81c6f93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:51.410846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.411514 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5bdd8db0-31de-4740-b282-95a9ec28aeff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.412994 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1397/2806] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:51 2022] DELETE /identity/v3/projects/91ac53e0baa8457298bf54bd81c6f93f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:51.418669 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4f14-15f3-4dce-b0b3-58ff7b26935e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.418950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4f14-15f3-4dce-b0b3-58ff7b26935e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.419241 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4f14-15f3-4dce-b0b3-58ff7b26935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.439846 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a3cd4f14-15f3-4dce-b0b3-58ff7b26935e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:51.441192 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a3cd4f14-15f3-4dce-b0b3-58ff7b26935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:51.445220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffa47f98c5040c9a6805c3fdbbc9f91'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.504122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1398/2807] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:51.512333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.512602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.512877 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.526647 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffa47f98c5040c9a6805c3fdbbc9f91'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.527384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffa47f98c5040c9a6805c3fdbbc9f91'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.528029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffa47f98c5040c9a6805c3fdbbc9f91'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.531906 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:51.533103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:51.537289 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffa47f98c5040c9a6805c3fdbbc9f91'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:51.542902 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1399/2808] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:51.550463 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.556591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1410/2809] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:51 2022] DELETE /identity/v3/users/6ffa47f98c5040c9a6805c3fdbbc9f91 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:51.560187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85536-a76f-4b0c-a3aa-be42ac12c5cb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:51.618694 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:51.619352 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.619639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.619918 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.620572 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:51.620876 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:51.622475 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.634623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1400/2810] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:24:51.638960 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.648959 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88468ca-546f-4a8b-aab1-013ec17fa025 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:51.654178 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:51.654777 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.655077 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.655360 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.656818 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:51.657905 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.758750 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1411/2811] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:51 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:51.760415 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.771289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fec13-7430-40c4-b17a-26ee9745f690 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:51.836829 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:51.837617 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.838025 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.838408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.839323 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:51.841433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.854757 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:51.953217 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1401/2812] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:51 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:51.957408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.967866 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c5b4e-2b73-4bd5-b2ec-f1deda4bd513 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:51.973476 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:51.977538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:51.977538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:51.977538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:51.977538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:51.977538 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:51.980331 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.982289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:51.989422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1412/2813] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:51.993435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:51.996230 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b57ef-1164-4ee7-bfd8-6091bb4b42a3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.001680 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.002216 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.002497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.002761 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] PATH_INFO: `/v3/projects/00b64e3bf200401bb5925faedbb2b276` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.003982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b64e3bf200401bb5925faedbb2b276)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.005039 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92629cfb-6b17-47df-8018-89daa7fcb402 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.006971 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.012438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.012438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.012438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.012438 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.012980 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.013278 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.023734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1413/2814] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:52.028170 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.038935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39329-6d96-4e80-b9a2-b4c011f0f3bb admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.043720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321762e-c8be-49f3-aac3-75afc45b03a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.044365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321762e-c8be-49f3-aac3-75afc45b03a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.044648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321762e-c8be-49f3-aac3-75afc45b03a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.045037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321762e-c8be-49f3-aac3-75afc45b03a4 admin admin] PATH_INFO: `/v3/projects/49dc741cacd14927aeb30024ff6c23c6/users/efe2fe44fb2a45ef8d3d2caa9f93634a/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.055012 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:52.061429 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321762e-c8be-49f3-aac3-75afc45b03a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49dc741cacd14927aeb30024ff6c23c6, user_id=efe2fe44fb2a45ef8d3d2caa9f93634a, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.064125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321762e-c8be-49f3-aac3-75afc45b03a4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.153370 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b64e3bf200401bb5925faedbb2b276', 'request_id': 'req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:52.155224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1414/2815] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:52 2022] PUT /identity/v3/projects/49dc741cacd14927aeb30024ff6c23c6/users/efe2fe44fb2a45ef8d3d2caa9f93634a/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:52.157094 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.157388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.157670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.161401 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b64e3bf200401bb5925faedbb2b276', 'request_id': 'req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:52.188064 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] MFA Rules not processed for user `efe2fe44fb2a45ef8d3d2caa9f93634a`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:52.189314 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:52.211996 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b64e3bf200401bb5925faedbb2b276', 'request_id': 'req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:52.226164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] The token cache is being invalidate because project 00b64e3bf200401bb5925faedbb2b276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:52.226454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:52.227123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2c5f9b1f-58b6-4e62-9eaf-2786d517eafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:52.229300 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1402/2816] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:51 2022] DELETE /identity/v3/projects/00b64e3bf200401bb5925faedbb2b276 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:52.235312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aecaa-1fc9-4cc8-9d6e-062623e8f827 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.235702 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aecaa-1fc9-4cc8-9d6e-062623e8f827 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.236042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aecaa-1fc9-4cc8-9d6e-062623e8f827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.263800 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1415/2817] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:52.269314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-998aecaa-1fc9-4cc8-9d6e-062623e8f827 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:52.271178 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-998aecaa-1fc9-4cc8-9d6e-062623e8f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:52.273597 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.274882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:52.334166 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1403/2818] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:52.342161 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.342450 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.342729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.345408 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fde3db-b13f-44fd-9f04-e70032d70582 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.350235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.351839 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.352132 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.352403 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.353029 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.354917 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.361988 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:52.363236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:52.371875 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1404/2819] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:52.375795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.387316 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474cc85-efe4-434e-927a-4ed81bdb300f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.392236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.392993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.393123 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.393394 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.394072 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:52.394390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.396601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.408733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1405/2820] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 10:24:52.412923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.422390 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa235b-3322-4f5a-95b8-e42edded853e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.427100 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.427703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.427984 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.428255 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.428871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.430736 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.475203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1416/2821] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:52.479298 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.543741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1406/2822] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:52.544963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.546179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358778-ebb0-43a6-8394-b6693cc00544 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.552203 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:52.581043 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:52.606078 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.606650 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.607008 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.607325 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.608109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.609438 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c820-1ec8-41a1-8420-528f24917287 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.610235 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.616891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.617590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.618005 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.618374 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.619138 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.621279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.684641 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1417/2823] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/users => generated 298 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:52.688424 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.698357 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6bb64-6435-4932-bfc2-bc67e7caf763 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.703133 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.703736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.704016 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.704291 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.704919 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.706557 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.717103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1418/2824] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:24:52.720939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.730272 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770005c-16fc-4e63-9f51-3851f96a91f2 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.736872 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.737826 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.738184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.739066 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] PATH_INFO: `/v3/projects/dcb97e0c5f984dafa1aad304f73015bf/users/13ac1099d2b94e93bf53de80d36f5382/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.743647 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1407/2825] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:52.748774 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.755730 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb97e0c5f984dafa1aad304f73015bf, user_id=13ac1099d2b94e93bf53de80d36f5382, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.758494 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.759533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73969735-4f7f-48ad-861f-c0af267da256 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.764373 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.764983 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.765261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.765534 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.766167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.767776 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.778056 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1408/2826] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:52.781790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.791155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2535-86ab-4070-81d9-0c466c6aa41a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.795759 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.796317 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.796623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.796900 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.797515 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.799051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.809851 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1409/2827] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:24:52.813594 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.823555 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdd84f-5be5-4b00-ac4d-14fce12c090f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.828353 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469ccc2-c96d-459c-872a-d787da9432d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.828958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ccc2-c96d-459c-872a-d787da9432d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.829240 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ccc2-c96d-459c-872a-d787da9432d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.829512 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ccc2-c96d-459c-872a-d787da9432d9 admin admin] PATH_INFO: `/v3/projects/b9fa2ecc3d514c8099fdea9af1f97aad/users/8fd75db89c304716ab3d6d64ae16072d/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.847442 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469ccc2-c96d-459c-872a-d787da9432d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fa2ecc3d514c8099fdea9af1f97aad, user_id=8fd75db89c304716ab3d6d64ae16072d, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.850505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1419/2828] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:52 2022] PUT /identity/v3/projects/dcb97e0c5f984dafa1aad304f73015bf/users/13ac1099d2b94e93bf53de80d36f5382/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:52.853426 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469ccc2-c96d-459c-872a-d787da9432d9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.854505 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.921582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a43709-2012-451c-8377-83897b6c1113 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.927581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.928196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.928482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.928763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.929537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.931270 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.941658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1420/2829] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:52.945791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:52.955840 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee075ed-eb6c-4d32-9045-d574be9ad880 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:52.960594 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f74fb-459b-48f6-aca6-b05406d8f4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:52.964573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f74fb-459b-48f6-aca6-b05406d8f4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.964573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f74fb-459b-48f6-aca6-b05406d8f4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.964573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f74fb-459b-48f6-aca6-b05406d8f4bf admin admin] PATH_INFO: `/v3/projects/dcb97e0c5f984dafa1aad304f73015bf/users/13ac1099d2b94e93bf53de80d36f5382/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.964573 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f74fb-459b-48f6-aca6-b05406d8f4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb97e0c5f984dafa1aad304f73015bf, user_id=13ac1099d2b94e93bf53de80d36f5382, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:52.966440 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f74fb-459b-48f6-aca6-b05406d8f4bf admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:52.968894 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1410/2830] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:52 2022] PUT /identity/v3/projects/b9fa2ecc3d514c8099fdea9af1f97aad/users/8fd75db89c304716ab3d6d64ae16072d/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:52.980200 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:52.980447 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:52.980721 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:52.993055 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1421/2831] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:52 2022] PUT /identity/v3/projects/dcb97e0c5f984dafa1aad304f73015bf/users/13ac1099d2b94e93bf53de80d36f5382/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:53.003098 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.003415 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.003713 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.014653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] MFA Rules not processed for user `8fd75db89c304716ab3d6d64ae16072d`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:53.015958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:53.039531 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] MFA Rules not processed for user `13ac1099d2b94e93bf53de80d36f5382`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:53.041089 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:53.080525 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:53.086843 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1411/2832] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:53.091315 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.116511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1422/2833] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:53.128405 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.129842 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:53.161360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eace59f-6ffb-4bad-b43c-bdd0a16650b1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.166278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.166891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.170762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.170762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.170762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.170762 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.182582 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:53.201053 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40d70a-665d-4a01-afcd-1ae690fa9aa5 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.204736 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.205388 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.205673 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.206097 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.215088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.216633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.227593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1423/2834] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:24:53.286802 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1412/2835] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:53 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 10:24:53.290943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.302060 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27cce6-0cd4-46c7-a262-ae79ac5babcb service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.359909 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.360527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.360812 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.361086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.361735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.364176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.404473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.454513 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1424/2836] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:53 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:53.460332 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.465703 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf78293-b287-4380-80b5-0f55250dadab admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.469433 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.470050 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.470273 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7ee9b-584f-4342-9622-537fd46cee51 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.470378 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.470593 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.475911 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.476516 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.476802 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.477109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.477993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.491798 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.514044 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1425/2837] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:24:53.517888 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.529372 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919bc4-cad3-4ce2-adfe-20d4c6ee7ca4 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.534129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.534614 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.534779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.535082 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.535393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] PATH_INFO: `/v3/projects/3afae5b6595a4a009ed24ee2abbceaa9/users/87bdbb078a524cd8a5c399826a2ea1be/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.536238 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.545763 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1413/2838] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:24:53.559907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afae5b6595a4a009ed24ee2abbceaa9, user_id=87bdbb078a524cd8a5c399826a2ea1be, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.559907 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.640861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1426/2839] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:53 2022] PUT /identity/v3/projects/3afae5b6595a4a009ed24ee2abbceaa9/users/87bdbb078a524cd8a5c399826a2ea1be/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:53.644856 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.655922 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e802e2-0989-414b-85fe-a2e835ac2ad8 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.712975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9150e7-fda1-4df1-8bbc-3185144a5c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.713502 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9150e7-fda1-4df1-8bbc-3185144a5c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.713793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9150e7-fda1-4df1-8bbc-3185144a5c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.714061 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9150e7-fda1-4df1-8bbc-3185144a5c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.714705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9150e7-fda1-4df1-8bbc-3185144a5c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.716324 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9150e7-fda1-4df1-8bbc-3185144a5c7c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.729933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1414/2840] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:53.731138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.742836 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391187-1b1a-4297-ae87-7336b01bdfbd admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.750952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.750952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.750952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.750952 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] PATH_INFO: `/v3/projects/3afae5b6595a4a009ed24ee2abbceaa9/users/87bdbb078a524cd8a5c399826a2ea1be/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.751361 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afae5b6595a4a009ed24ee2abbceaa9, user_id=87bdbb078a524cd8a5c399826a2ea1be, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.753582 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.764373 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1427/2841] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:53 2022] PUT /identity/v3/projects/3afae5b6595a4a009ed24ee2abbceaa9/users/87bdbb078a524cd8a5c399826a2ea1be/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:53.776644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.776644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.776644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.806454 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] MFA Rules not processed for user `87bdbb078a524cd8a5c399826a2ea1be`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:53.807700 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:53.837529 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:53.876064 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1415/2842] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:53.886669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:53.888392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:53.907935 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:53.949612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c13c3-72d9-4339-b771-b6f64c4207c1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:53.953397 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:53.954001 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:53.954286 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:53.954580 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:53.963680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:53.965037 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:53.975434 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1428/2843] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:54.058527 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:54.079311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:54.088231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ae06-bbfc-470d-bb2b-e6dbe56aa035 service cinder] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:54.092474 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307de745-917c-47bf-95e5-9abbf89009ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:54.093155 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307de745-917c-47bf-95e5-9abbf89009ac service cinder] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:54.093520 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307de745-917c-47bf-95e5-9abbf89009ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:54.093882 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307de745-917c-47bf-95e5-9abbf89009ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:54.124503 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd716fb-c91c-41cf-a781-59e357017af4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:54.129137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:54.129768 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:54.130047 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:54.130322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] PATH_INFO: `/v3/users/efe2fe44fb2a45ef8d3d2caa9f93634a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:54.131496 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe2fe44fb2a45ef8d3d2caa9f93634a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:54.133406 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:54.153190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307de745-917c-47bf-95e5-9abbf89009ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:54.154628 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307de745-917c-47bf-95e5-9abbf89009ac service cinder] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:54.164731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1429/2844] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:24:54.333269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe2fe44fb2a45ef8d3d2caa9f93634a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:54.440892 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe2fe44fb2a45ef8d3d2caa9f93634a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:54.441281 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe2fe44fb2a45ef8d3d2caa9f93634a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:54.442483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe2fe44fb2a45ef8d3d2caa9f93634a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:54.449977 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe2fe44fb2a45ef8d3d2caa9f93634a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:54.469082 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1416/2845] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:54 2022] DELETE /identity/v3/users/efe2fe44fb2a45ef8d3d2caa9f93634a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:54.473757 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:54.474202 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:54.474776 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:54.494230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:54.495490 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:54.564046 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1430/2846] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:54.575658 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:54.577195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:54.637368 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:54.642972 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e1bbb-1fc7-4889-b6b1-af04a9d78d05 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:54.647305 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:54.647923 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:54.648201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:54.648471 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:54.659212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:54.660715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:54.670416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1417/2847] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:24:55.128425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:55.130993 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:55.143330 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebef16-877b-4596-999c-426dfebb92c9 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:55.148308 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:55.148756 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:55.149092 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:55.149365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] PATH_INFO: `/v3/projects/49dc741cacd14927aeb30024ff6c23c6` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:55.150527 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49dc741cacd14927aeb30024ff6c23c6)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:55.152697 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:55.263920 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49dc741cacd14927aeb30024ff6c23c6', 'request_id': 'req-2e53e734-eb70-449f-a0b8-40ada117959a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.265763 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:55.270323 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49dc741cacd14927aeb30024ff6c23c6', 'request_id': 'req-2e53e734-eb70-449f-a0b8-40ada117959a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.337555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49dc741cacd14927aeb30024ff6c23c6', 'request_id': 'req-2e53e734-eb70-449f-a0b8-40ada117959a'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.349120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] The token cache is being invalidate because project 49dc741cacd14927aeb30024ff6c23c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:55.349406 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.350024 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.351772 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1431/2848] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:55 2022] DELETE /identity/v3/projects/49dc741cacd14927aeb30024ff6c23c6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:55.356286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:55.429169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ca50-1a4e-40bc-9b6e-22a960586198 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:55.432413 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:55.433169 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:55.433456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:55.433735 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] PATH_INFO: `/v3/users/13ac1099d2b94e93bf53de80d36f5382` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:55.434879 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ac1099d2b94e93bf53de80d36f5382)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:55.436833 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:55.672881 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ac1099d2b94e93bf53de80d36f5382'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.737602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ac1099d2b94e93bf53de80d36f5382'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.738105 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ac1099d2b94e93bf53de80d36f5382'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.738725 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ac1099d2b94e93bf53de80d36f5382'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.746633 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1c8abb52-a2bc-45f1-be46-31974fca9f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ac1099d2b94e93bf53de80d36f5382'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:55.764622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1418/2849] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:24:55 2022] DELETE /identity/v3/users/13ac1099d2b94e93bf53de80d36f5382 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:24:56.259246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:56.260680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:56.274339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53e734-eb70-449f-a0b8-40ada117959a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:56.334861 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:56.335567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:56.335847 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:56.336120 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] PATH_INFO: `/v3/projects/dcb97e0c5f984dafa1aad304f73015bf` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:56.337238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb97e0c5f984dafa1aad304f73015bf)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:56.339156 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:56.454423 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb97e0c5f984dafa1aad304f73015bf', 'request_id': 'req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:56.460615 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb97e0c5f984dafa1aad304f73015bf', 'request_id': 'req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:56.503322 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb97e0c5f984dafa1aad304f73015bf', 'request_id': 'req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:56.514540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] The token cache is being invalidate because project dcb97e0c5f984dafa1aad304f73015bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:24:56.515341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:56.515479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-e1eea604-ebfc-44b2-b492-d25e3a0cd7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:24:56.516939 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1432/2850] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:24:56 2022] DELETE /identity/v3/projects/dcb97e0c5f984dafa1aad304f73015bf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:56.523819 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:56.524312 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:56.524432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:56.546164 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:56.547411 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:56.618301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1419/2851] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:56.626285 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:56.626574 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:56.626848 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:56.646943 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:56.648248 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:56.657158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1433/2852] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:24:56.662097 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:56.672116 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39b40-3f46-4360-b5eb-5bb017e1d9e6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:56.677482 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:56.678176 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:56.678483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:56.678724 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:56.679395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:24:56.679723 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:56.681286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:56.683908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:56.694967 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1420/2853] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:24:56.698639 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:56.708779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffd742-4f28-4412-9581-4be1813aa31a None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:56.713501 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:56.715290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:56.715290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:56.715290 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:56.715613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:56.717220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:56.820351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1434/2854] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:24:56 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:56.824575 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:56.836113 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0741-5c65-43f0-b181-a15f82f6dea9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:56.904121 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:56.904910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:56.905283 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:56.905832 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:56.906912 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:56.909619 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:56.982206 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1421/2855] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:24:56 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:24:56.983100 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:56.998447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233868a2-845f-442c-b7b3-75457ea09be1 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:57.006504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:57.007216 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:57.007511 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:57.008447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:57.009217 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:57.011569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:57.025009 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1435/2856] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:57.030265 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:57.043742 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a5bb-a9b9-4b8c-bbe3-0c7872f77f72 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:57.048916 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97eaaf-3e10-4ec8-bc7e-77650e5c74d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:57.049653 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97eaaf-3e10-4ec8-bc7e-77650e5c74d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:57.050055 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97eaaf-3e10-4ec8-bc7e-77650e5c74d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:57.050533 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97eaaf-3e10-4ec8-bc7e-77650e5c74d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:57.051262 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97eaaf-3e10-4ec8-bc7e-77650e5c74d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:57.054029 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97eaaf-3e10-4ec8-bc7e-77650e5c74d2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:57.071622 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1422/2857] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:24:57.073466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:57.098342 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4038-8a31-45a8-9c28-d68baf0ec64c admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:57.104791 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:57.105596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:57.109658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:57.109658 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] PATH_INFO: `/v3/projects/d3610ef4e07548e1a88d76356d8170a6/users/31a26b7651fd463393526fab64ccf1c7/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:57.124056 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3610ef4e07548e1a88d76356d8170a6, user_id=31a26b7651fd463393526fab64ccf1c7, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:57.126782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:57.230545 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1436/2858] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:24:57 2022] PUT /identity/v3/projects/d3610ef4e07548e1a88d76356d8170a6/users/31a26b7651fd463393526fab64ccf1c7/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:24:57.235844 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:57.236139 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:57.236418 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:57.256668 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:24:57.257895 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:57.322741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1423/2859] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:24:57.330499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:24:57.332612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:24:57.343409 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:24:57.393790 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875f71-a796-4c1f-bea8-5570f3960418 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:24:57.398129 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26fbd-3375-4dfc-be2d-89f38abab8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:24:57.398734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26fbd-3375-4dfc-be2d-89f38abab8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:24:57.399014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26fbd-3375-4dfc-be2d-89f38abab8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:24:57.399339 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26fbd-3375-4dfc-be2d-89f38abab8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:24:57.410693 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c26fbd-3375-4dfc-be2d-89f38abab8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:24:57.412637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c26fbd-3375-4dfc-be2d-89f38abab8fa service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:24:57.422074 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1437/2860] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:01.635448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:01.637253 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:01.651051 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f6d9c-b37f-4cff-94d2-fe1f6fafc570 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:01.657771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:01.658289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:01.661811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:01.661811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] PATH_INFO: `/v3/users/8fd75db89c304716ab3d6d64ae16072d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:01.661811 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd75db89c304716ab3d6d64ae16072d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:01.662743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:01.890644 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd75db89c304716ab3d6d64ae16072d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:01.978519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd75db89c304716ab3d6d64ae16072d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:01.979932 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd75db89c304716ab3d6d64ae16072d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:01.980926 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd75db89c304716ab3d6d64ae16072d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:01.998122 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd75db89c304716ab3d6d64ae16072d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:02.002376 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:02.016146 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1424/2861] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:25:01 2022] DELETE /identity/v3/users/8fd75db89c304716ab3d6d64ae16072d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:02.021252 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:02.021740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:02.021989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:02.045979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:02.047351 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:02.113206 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1438/2862] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:02.128813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:02.130973 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:02.233980 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8345d-4ef1-428a-b1c0-e2225edd9805 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:02.243293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:02.243293 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:02.243507 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:02.243771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:02.265679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:02.265679 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:02.274333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1425/2863] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:02.778652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:02.778652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:02.792956 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14a68-e934-4323-b584-0d1da36db75b None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:02.805963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:02.810834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:02.810834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:02.810834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] PATH_INFO: `/v3/projects/b9fa2ecc3d514c8099fdea9af1f97aad` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:02.810834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fa2ecc3d514c8099fdea9af1f97aad)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:02.810834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:02.935610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fa2ecc3d514c8099fdea9af1f97aad', 'request_id': 'req-c8e515f8-a631-4c66-9b30-0fab1a5188ab'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:02.942785 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fa2ecc3d514c8099fdea9af1f97aad', 'request_id': 'req-c8e515f8-a631-4c66-9b30-0fab1a5188ab'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.057224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fa2ecc3d514c8099fdea9af1f97aad', 'request_id': 'req-c8e515f8-a631-4c66-9b30-0fab1a5188ab'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.077957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] The token cache is being invalidate because project b9fa2ecc3d514c8099fdea9af1f97aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:25:03.078255 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.079548 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-c8e515f8-a631-4c66-9b30-0fab1a5188ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.082652 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1439/2864] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:25:02 2022] DELETE /identity/v3/projects/b9fa2ecc3d514c8099fdea9af1f97aad => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:03.087037 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:03.100220 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63989a66-e254-4160-90ed-bf2399a87889 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:03.248351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:03.249009 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:03.249289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:03.249562 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] PATH_INFO: `/v3/users/87bdbb078a524cd8a5c399826a2ea1be` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:03.250937 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87bdbb078a524cd8a5c399826a2ea1be)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:03.261531 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:03.392596 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:03.392877 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:03.393246 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:03.426572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] MFA Rules not processed for user `31a26b7651fd463393526fab64ccf1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:03.427984 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:03.450792 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:03.505109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1440/2865] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:25:03.509794 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:03.511187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:03.526939 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87bdbb078a524cd8a5c399826a2ea1be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.528913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53d910-7999-43ce-bc3b-5cd13c582427 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:03.534353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:03.534975 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:03.535319 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:03.536471 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:03.537139 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:03.539633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:03.627975 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87bdbb078a524cd8a5c399826a2ea1be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.628808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87bdbb078a524cd8a5c399826a2ea1be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.629790 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87bdbb078a524cd8a5c399826a2ea1be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.636320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1441/2866] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:25:03 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:25:03.640705 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87bdbb078a524cd8a5c399826a2ea1be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:03.643521 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:03.646079 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:03.668891 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1426/2867] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:25:03 2022] DELETE /identity/v3/users/87bdbb078a524cd8a5c399826a2ea1be => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:03.743463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8731e7-27c2-4d28-a81c-10f99f625387 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:03.803407 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:03.803954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:03.804304 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:03.804556 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:03.805197 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:03.807108 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:03.926901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1442/2868] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:25:03 2022] POST /identity/v3/users => generated 299 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:25:03.931132 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:03.943753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ababd-b4f1-4020-b94c-50b40c434381 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:03.946933 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:03.947602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:03.947910 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:03.948189 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:03.948963 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:03.950448 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:03.952908 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:03.961339 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1427/2869] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:25:03.965463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:03.977321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6b49e-ad7c-4190-8102-cf3b5e1950e0 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:03.982654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:03.983435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:03.983786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:03.984134 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] PATH_INFO: `/v3/projects/3b07517680644824bf4a6e622adc4960/users/bc8b8efb205d4001a9085ef1265bab12/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.000199 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b07517680644824bf4a6e622adc4960, user_id=bc8b8efb205d4001a9085ef1265bab12, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.002869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.090211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1443/2870] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:25:03 2022] PUT /identity/v3/projects/3b07517680644824bf4a6e622adc4960/users/bc8b8efb205d4001a9085ef1265bab12/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:04.094152 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.104681 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7e140-fb83-43bb-953c-4f756159ca3f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.119950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.121810 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:04.142166 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:04.181354 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.181976 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.182259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.182535 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.183197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.184808 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.195201 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1428/2871] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:25:04.198979 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b416169-6304-4903-b9c8-3bccd5fffbab admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.199486 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.204179 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.209352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.209352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.209352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] PATH_INFO: `/v3/projects/3afae5b6595a4a009ed24ee2abbceaa9` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.209352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afae5b6595a4a009ed24ee2abbceaa9)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.209352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.214059 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717bf0b-adda-494f-8f54-521af72ad902 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.220157 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8419b8-0f3f-48a2-914c-955ef125f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.220935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8419b8-0f3f-48a2-914c-955ef125f9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.221227 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8419b8-0f3f-48a2-914c-955ef125f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.221504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8419b8-0f3f-48a2-914c-955ef125f9d0 admin admin] PATH_INFO: `/v3/projects/3b07517680644824bf4a6e622adc4960/users/bc8b8efb205d4001a9085ef1265bab12/roles/435125566e954d20a3b8273b7c806b98` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.224199 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8419b8-0f3f-48a2-914c-955ef125f9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b07517680644824bf4a6e622adc4960, user_id=bc8b8efb205d4001a9085ef1265bab12, role_id=435125566e954d20a3b8273b7c806b98)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.227249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8419b8-0f3f-48a2-914c-955ef125f9d0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.244408 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1429/2872] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:25:04 2022] PUT /identity/v3/projects/3b07517680644824bf4a6e622adc4960/users/bc8b8efb205d4001a9085ef1265bab12/roles/435125566e954d20a3b8273b7c806b98 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:25:04.346950 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afae5b6595a4a009ed24ee2abbceaa9', 'request_id': 'req-684afa43-a8ba-4baa-a71f-cb064f605d14'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:04.352019 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afae5b6595a4a009ed24ee2abbceaa9', 'request_id': 'req-684afa43-a8ba-4baa-a71f-cb064f605d14'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:04.423543 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afae5b6595a4a009ed24ee2abbceaa9', 'request_id': 'req-684afa43-a8ba-4baa-a71f-cb064f605d14'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:04.436734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] The token cache is being invalidate because project 3afae5b6595a4a009ed24ee2abbceaa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:25:04.437143 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:04.438047 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-684afa43-a8ba-4baa-a71f-cb064f605d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:04.443660 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1444/2873] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:25:04 2022] DELETE /identity/v3/projects/3afae5b6595a4a009ed24ee2abbceaa9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:25:04.445521 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.445882 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.446236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.467713 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:04.469046 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:04.539885 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1430/2874] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:25:04.546579 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.547205 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.547589 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.574742 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:04.576013 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:04.586177 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1445/2875] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:04.589214 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.606254 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83026-e84a-4bb3-b091-7bcf6f4e74fc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.609946 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.610942 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.611333 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.611695 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.612462 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:25:04.613038 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.615351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.644701 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1431/2876] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:25:04.646350 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.656071 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dad84-3fbb-4d58-ad23-16d3a6d94d2e None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.662820 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.664115 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.664455 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.664951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.667256 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.671227 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.796482 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1446/2877] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:25:04 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:25:04.797685 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.808246 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449b031-a62a-472e-812e-fbbca5710c04 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.862698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:04.862698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:04.884249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.896461 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.896752 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.897510 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.898187 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:04.903330 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:04.930659 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274c00-32d0-42c1-ad84-f051015dacc5 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:04.934779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:04.935454 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:04.935957 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:04.936464 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:04.938955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:04.974719 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:05.008653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:05.010182 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:05.020815 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1447/2878] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 10:25:05.023986 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:05.034952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1432/2879] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:25:04 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:25:05.037958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:05.048987 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287027-d7c2-40b9-ad90-c4555c6d118d service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:05.054069 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:05.054682 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:05.054963 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:05.055274 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:05.056080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:05.057683 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:05.108989 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1448/2880] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:25:05.112691 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:05.128416 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7130-2d66-4da7-b526-4ecfddee5b43 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:05.133523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f08799-f102-424d-866a-a29ee4e1e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:05.134143 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f08799-f102-424d-866a-a29ee4e1e740 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:05.134421 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f08799-f102-424d-866a-a29ee4e1e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:05.134698 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f08799-f102-424d-866a-a29ee4e1e740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:05.135596 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f08799-f102-424d-866a-a29ee4e1e740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:05.136974 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f08799-f102-424d-866a-a29ee4e1e740 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:05.152623 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1433/2881] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 10:25:05.153537 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:05.163468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e1915-0926-4bcb-b20b-ac8e8dfc7258 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:05.168122 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:05.168783 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] REQUEST_METHOD: `PUT` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:05.169084 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:05.169340 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] PATH_INFO: `/v3/projects/551e67a3a9c04bada2aaafa0ba5b51b4/users/e37caf8215304508a2790bd11e5e4a50/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:05.184002 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=551e67a3a9c04bada2aaafa0ba5b51b4, user_id=e37caf8215304508a2790bd11e5e4a50, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:05.186740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:05.305886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1449/2882] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:25:05 2022] PUT /identity/v3/projects/551e67a3a9c04bada2aaafa0ba5b51b4/users/e37caf8215304508a2790bd11e5e4a50/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:25:05.317787 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f60b9-edb5-4bcb-b9cf-03699f1975fe None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:05.319497 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f60b9-edb5-4bcb-b9cf-03699f1975fe None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:05.320035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f60b9-edb5-4bcb-b9cf-03699f1975fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:05.356095 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6d9f60b9-edb5-4bcb-b9cf-03699f1975fe None None] MFA Rules not processed for user `e37caf8215304508a2790bd11e5e4a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:05.357868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6d9f60b9-edb5-4bcb-b9cf-03699f1975fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:05.450210 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1434/2883] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:25:05.461555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:05.463369 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:05.527610 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4a30-0495-4ad4-bfaa-8ddfde72c65e admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:05.532243 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:05.533190 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:05.534345 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:05.534695 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:05.544636 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:05.547006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:05.557312 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1450/2884] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:11.062219 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29172daf-6963-4f3b-9df3-b24181614660 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:11.062706 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29172daf-6963-4f3b-9df3-b24181614660 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:11.062823 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29172daf-6963-4f3b-9df3-b24181614660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:11.103641 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-29172daf-6963-4f3b-9df3-b24181614660 None None] MFA Rules not processed for user `bc8b8efb205d4001a9085ef1265bab12`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:11.105616 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-29172daf-6963-4f3b-9df3-b24181614660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:11.189602 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1435/2885] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:11.202344 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:11.219600 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab85aa-797f-4526-8db0-e697d0cf4e71 service glance] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:11.225277 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5950076b981b43feba3185f1fb513729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:11.226103 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:11.226384 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:11.226700 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:11.228613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:11.303152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:11.304710 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:11.316327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1451/2886] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:11.647474 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224bd95-eaae-4ad0-9bd8-4bd842e62100 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:11.647758 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224bd95-eaae-4ad0-9bd8-4bd842e62100 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:11.648204 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224bd95-eaae-4ad0-9bd8-4bd842e62100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:11.680218 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8224bd95-eaae-4ad0-9bd8-4bd842e62100 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:11.681351 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8224bd95-eaae-4ad0-9bd8-4bd842e62100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:11.737993 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:11.746608 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1436/2887] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:11.908426 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:11.909933 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:11.958586 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:11.997540 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d7699-da40-409b-b118-9e18e24528e3 service glance] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:12.003204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:12.004964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:12.004964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:12.004964 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:12.017569 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:12.019392 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:12.031261 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1452/2888] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:25:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:25:21.857035 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:22.074653 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=18 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:22.842671 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb1673-673a-40c3-9edf-ed9f9e72cd70 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:22.842958 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb1673-673a-40c3-9edf-ed9f9e72cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:22.843274 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb1673-673a-40c3-9edf-ed9f9e72cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:22.865950 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-acbb1673-673a-40c3-9edf-ed9f9e72cd70 None None] MFA Rules not processed for user `31a26b7651fd463393526fab64ccf1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:22.867212 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-acbb1673-673a-40c3-9edf-ed9f9e72cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:22.874395 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1437/2889] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:26.717870 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:26.792946 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f943f5-4399-47e0-bf36-72048294ff9f service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:26.797966 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:26.798640 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:26.798924 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:26.799251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] PATH_INFO: `/v3/users/e37caf8215304508a2790bd11e5e4a50` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:26.800370 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37caf8215304508a2790bd11e5e4a50)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:26.802845 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:26.978734 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37caf8215304508a2790bd11e5e4a50'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:27.027954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37caf8215304508a2790bd11e5e4a50'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:27.028575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37caf8215304508a2790bd11e5e4a50'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:27.029207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37caf8215304508a2790bd11e5e4a50'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:27.036680 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37caf8215304508a2790bd11e5e4a50'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:27.055236 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1453/2890] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:25:26 2022] DELETE /identity/v3/users/e37caf8215304508a2790bd11e5e4a50 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:27.059788 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:27.060278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:27.060391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:27.079276 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:27.080473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:27.137268 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1438/2891] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:27.147231 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:27.148669 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:27.205869 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68e7ad-4269-4517-bc0d-57d9d57b7091 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:27.209581 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b8fe6-dc94-4ea3-b23e-dd13306a0b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:27.210188 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b8fe6-dc94-4ea3-b23e-dd13306a0b88 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:27.210469 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b8fe6-dc94-4ea3-b23e-dd13306a0b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:27.210769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b8fe6-dc94-4ea3-b23e-dd13306a0b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:27.219737 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b8fe6-dc94-4ea3-b23e-dd13306a0b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:27.221211 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b8fe6-dc94-4ea3-b23e-dd13306a0b88 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:27.230637 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1454/2892] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:27.807865 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:27.809359 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:27.824237 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ed58-8c45-4b7b-b200-dec5cd2bb609 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:27.829197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:27.829871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:27.830155 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:27.830431 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] PATH_INFO: `/v3/projects/551e67a3a9c04bada2aaafa0ba5b51b4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:27.831629 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551e67a3a9c04bada2aaafa0ba5b51b4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:27.833715 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:27.972259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551e67a3a9c04bada2aaafa0ba5b51b4', 'request_id': 'req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:27.979566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551e67a3a9c04bada2aaafa0ba5b51b4', 'request_id': 'req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:28.034195 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551e67a3a9c04bada2aaafa0ba5b51b4', 'request_id': 'req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:28.046279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] The token cache is being invalidate because project 551e67a3a9c04bada2aaafa0ba5b51b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:25:28.046693 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:28.047440 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0645bb7f-17cc-40d6-be39-d3cf78c7dae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:25:28.049115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1439/2893] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:25:27 2022] DELETE /identity/v3/projects/551e67a3a9c04bada2aaafa0ba5b51b4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:28.054153 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.054609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.054609 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.074983 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:28.076376 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:28.094309 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:28.138657 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1455/2894] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:28.147020 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d82775-6039-4530-9734-395f9c787893 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.147532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d82775-6039-4530-9734-395f9c787893 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.147532 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d82775-6039-4530-9734-395f9c787893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.167360 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-20d82775-6039-4530-9734-395f9c787893 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:28.168565 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-20d82775-6039-4530-9734-395f9c787893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:28.176425 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1440/2895] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:28.180567 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.190099 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314934-34c1-4a95-984b-3e57fb5692eb None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.195731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.196348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.196612 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.196886 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.197530 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81736) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 10:25:28.197837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.199467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.211729 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1456/2896] 10.222.0.62 () {58 vars in 1153 bytes} [Tue Sep 13 10:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 10:25:28.221464 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d82775-6039-4530-9734-395f9c787893 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.231111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d82775-6039-4530-9734-395f9c787893 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.236197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.236800 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.237087 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.237364 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.237998 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.239981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.321250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1441/2897] 10.222.0.62 () {60 vars in 1153 bytes} [Tue Sep 13 10:25:28 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:25:28.325472 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.336306 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5363f-d1dd-477d-947f-511b977fe0ee admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.392613 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.393215 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.393892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.393892 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] PATH_INFO: `/v3/users` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.394364 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.397004 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.502593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1457/2898] 10.222.0.62 () {60 vars in 1144 bytes} [Tue Sep 13 10:25:28 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 10:25:28.507257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.517868 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394eb0-d438-4334-a2d1-5bb24c0d3047 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.522571 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.523183 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.523504 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.523797 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.524432 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.526261 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.535935 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1442/2899] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:25:28.540224 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.550313 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc9f71-a1bc-47b5-a485-61a13c59ca4a admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.555497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a60e6-079d-45b6-a3d8-21fd104ff098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.556093 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a60e6-079d-45b6-a3d8-21fd104ff098 admin admin] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.556378 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a60e6-079d-45b6-a3d8-21fd104ff098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.556654 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a60e6-079d-45b6-a3d8-21fd104ff098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.557296 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a60e6-079d-45b6-a3d8-21fd104ff098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.558884 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a60e6-079d-45b6-a3d8-21fd104ff098 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.570128 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1458/2900] 10.222.0.62 () {58 vars in 1122 bytes} [Tue Sep 13 10:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 10:25:28.574173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.584015 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479953d-d569-4fee-a39a-2bc3c8180442 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.588587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.589226 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.589506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.589775 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] PATH_INFO: `/v3/projects/501f1c67ddc54fddaa630db4f910a549/users/5bbb996f3c644578ad316b74a05fed44/roles/28d0ae899af34488a7f267d899b9aff7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.603982 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] RBAC: Authorizing `identity:create_grant(project_id=501f1c67ddc54fddaa630db4f910a549, user_id=5bbb996f3c644578ad316b74a05fed44, role_id=28d0ae899af34488a7f267d899b9aff7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.606601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.690904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1443/2901] 10.222.0.62 () {60 vars in 1483 bytes} [Tue Sep 13 10:25:28 2022] PUT /identity/v3/projects/501f1c67ddc54fddaa630db4f910a549/users/5bbb996f3c644578ad316b74a05fed44/roles/28d0ae899af34488a7f267d899b9aff7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:25:28.701725 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bfe63-da05-498d-9e13-15af30952109 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.702015 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bfe63-da05-498d-9e13-15af30952109 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.702289 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bfe63-da05-498d-9e13-15af30952109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.729955 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-157bfe63-da05-498d-9e13-15af30952109 None None] MFA Rules not processed for user `5bbb996f3c644578ad316b74a05fed44`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:28.731207 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-157bfe63-da05-498d-9e13-15af30952109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:28.794187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1459/2902] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 10:25:28.805850 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:28.807600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:28.821990 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:28.870301 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0d753-9b3f-4d77-8d0b-17644e68a65a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:28.874350 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566f088827ab4710bf54d64415831a1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:28.875205 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:28.875528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:28.875784 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:28.887271 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:28.889424 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:28.900820 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1444/2903] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:29.018771 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f7670-13a7-46fd-84ca-91a1c1810b30 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:29.021204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f7670-13a7-46fd-84ca-91a1c1810b30 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:29.021204 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f7670-13a7-46fd-84ca-91a1c1810b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:29.044593 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-9e9f7670-13a7-46fd-84ca-91a1c1810b30 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:29.044767 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-9e9f7670-13a7-46fd-84ca-91a1c1810b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:29.058462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:29.106962 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1460/2904] 10.222.0.62 () {58 vars in 1009 bytes} [Tue Sep 13 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:29.120286 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:29.121717 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:29.204747 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f999d-d9c5-4234-ad6a-2b0b3e0c2f3e service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:29.210795 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:29.211445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:29.211718 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:29.211985 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:29.221211 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:29.222852 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:29.232771 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1445/2905] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:25:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:25:29.935954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] REQUEST_METHOD: `POST` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:29.936462 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:29.936912 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:29.957720 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.auth.core [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] MFA Rules not processed for user `9a6e7e1e625247a3963aa8a0a29eb478`. Rule list: `[]` (Enabled: `True`). {{(pid=81736) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:25:29.959096 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:30.033196 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1461/2906] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:25:39.230670 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=52 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:40.025607 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=42 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:40.098611 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:40.100590 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:40.157479 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a0ac-c4eb-490b-bb6d-d56c7df79a17 service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:40.161098 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:40.161699 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:40.161981 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:40.162257 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:40.164483 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:40.236566 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:40.238071 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:40.249595 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1446/2907] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:45.944447 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:45.954542 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9820b-5e0d-4d0c-8723-6be71ee31b85 None None] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:45.958158 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:45.958782 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:45.959043 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:45.959383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:45.961125 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:45.975510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:45.976947 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:45.988383 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1462/2908] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:25:50.244639 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=54 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:55.983715 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=6 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:25:58.925162 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:25:58.934491 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81364-9c70-4439-9975-e463dfe81330 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:25:58.938118 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:25:58.938883 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:25:58.939186 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:25:58.939476 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:25:58.944250 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:25:59.009335 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:25:59.011377 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9d887-4bc8-4b9b-b9a0-e13861fd4675 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:25:59.022073 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1447/2909] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:25:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:26:09.017473 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=30 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:26:16.172941 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:16.233953 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2837a0-7b17-4c2b-9250-559fb0060626 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:16.238458 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:16.239080 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:16.239398 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:16.239670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] PATH_INFO: `/v3/users/5bbb996f3c644578ad316b74a05fed44` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:16.240765 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bbb996f3c644578ad316b74a05fed44)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:16.242745 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:16.440519 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bbb996f3c644578ad316b74a05fed44'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:16.510354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bbb996f3c644578ad316b74a05fed44'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:16.511006 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bbb996f3c644578ad316b74a05fed44'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:16.511666 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bbb996f3c644578ad316b74a05fed44'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:16.519014 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bbb996f3c644578ad316b74a05fed44'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:16.537090 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1463/2910] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:26:16 2022] DELETE /identity/v3/users/5bbb996f3c644578ad316b74a05fed44 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:26:16.542243 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:16.542729 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:16.542834 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:16.563024 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] MFA Rules not processed for user `47dfc0e354ea47fc9cdf40b73108b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:26:16.564269 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:16.620528 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1448/2911] 10.222.0.62 () {58 vars in 958 bytes} [Tue Sep 13 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:26:16.630251 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:16.631698 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:16.691970 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5da74c-b3d8-40ca-a620-2afb31757858 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:16.695575 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:16.696176 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:16.696467 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:16.696740 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:16.705958 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:16.707422 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:16.716879 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1464/2912] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:26:17.171322 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:17.173997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:17.185780 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ed69a-98b4-4927-ac6c-4f5cd707a9d8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:17.190969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:17.191678 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:17.192014 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:17.192344 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] PATH_INFO: `/v3/projects/501f1c67ddc54fddaa630db4f910a549` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:17.193563 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=501f1c67ddc54fddaa630db4f910a549)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:17.195427 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:17.308825 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501f1c67ddc54fddaa630db4f910a549', 'request_id': 'req-86a05b08-d39e-491c-b113-b6920f55ca31'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:17.315096 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501f1c67ddc54fddaa630db4f910a549', 'request_id': 'req-86a05b08-d39e-491c-b113-b6920f55ca31'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:17.393314 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501f1c67ddc54fddaa630db4f910a549', 'request_id': 'req-86a05b08-d39e-491c-b113-b6920f55ca31'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:17.405328 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] The token cache is being invalidate because project 501f1c67ddc54fddaa630db4f910a549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:26:17.405636 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:17.406259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:17.407824 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1449/2913] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:26:17 2022] DELETE /identity/v3/projects/501f1c67ddc54fddaa630db4f910a549 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:26:26.712605 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:26:27.402135 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:26:38.862598 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:38.864086 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:38.875320 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a88d5-8a70-4856-961c-478cc0268ae8 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:38.925352 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:38.925951 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:38.926229 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:38.926497 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:38.928302 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:38.937504 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:38.938913 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:38.947779 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1465/2914] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:26:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:26:48.943493 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=26 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:26:51.124385 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:51.125917 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:51.224709 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05b08-d39e-491c-b113-b6920f55ca31 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:51.231383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:51.232236 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:51.232697 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:51.233137 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] PATH_INFO: `/v3/users/9a6e7e1e625247a3963aa8a0a29eb478` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:51.234445 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6e7e1e625247a3963aa8a0a29eb478)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:51.236600 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:51.360603 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:51.365354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:51.450510 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf021b-254a-41aa-9a2f-2d9ae6663c1b service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:51.456321 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:51.456934 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:51.457329 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:51.457591 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:51.459437 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:51.472436 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:51.474058 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:51.485699 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1466/2915] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:26:51.516444 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6e7e1e625247a3963aa8a0a29eb478'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:51.608952 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6e7e1e625247a3963aa8a0a29eb478'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:51.609604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6e7e1e625247a3963aa8a0a29eb478'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:51.610279 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6e7e1e625247a3963aa8a0a29eb478'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:51.618861 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6e7e1e625247a3963aa8a0a29eb478'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:51.639111 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1450/2916] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:26:51 2022] DELETE /identity/v3/users/9a6e7e1e625247a3963aa8a0a29eb478 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:26:51.911184 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:51.927599 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f8fb0-d302-417e-8082-9eb42461fb96 service nova] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:52.006862 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:52.007463 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:52.007751 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:52.008034 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] PATH_INFO: `/v3/projects/8879b3720c0d482d8118a0d2e5590b54` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:52.009354 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8879b3720c0d482d8118a0d2e5590b54)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:52.011355 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:52.114180 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8879b3720c0d482d8118a0d2e5590b54', 'request_id': 'req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:52.120817 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8879b3720c0d482d8118a0d2e5590b54', 'request_id': 'req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:52.186011 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8879b3720c0d482d8118a0d2e5590b54', 'request_id': 'req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:52.197466 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] The token cache is being invalidate because project 8879b3720c0d482d8118a0d2e5590b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:26:52.197901 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:52.198555 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:26:52.200088 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1467/2917] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:26:51 2022] DELETE /identity/v3/projects/8879b3720c0d482d8118a0d2e5590b54 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:26:53.772523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:26:53.774067 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:53.853136 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e4dc-1235-4253-9c3f-cb9d73a03542 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:26:53.866167 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:26:53.917592 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81eb5931848749a1ad22d9d228899cda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:26:53.918295 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:26:53.918578 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:26:53.918853 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:26:53.920786 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:26:53.933088 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:26:53.935259 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:26:53.946042 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1451/2918] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:26:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:27:02.194076 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:04.008673 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=26 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:05.007881 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:05.009311 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:05.067528 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b5c5c-d0a7-4695-acd5-186ad2ef9088 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:05.071230 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5889d3606c904dac9b4674efed1f94a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:05.071834 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:05.072118 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:05.072393 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:05.074136 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:05.135853 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:05.137421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:05.147220 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1468/2919] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:27:12.048612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:12.050173 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:12.064275 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7661d-8e96-438a-ab35-8a1f6fc65b91 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:12.068822 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:12.069451 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:12.069731 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:12.070004 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] PATH_INFO: `/v3/users/7ddb2572cac446aa92a0bcf170d9e8d6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:12.071115 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddb2572cac446aa92a0bcf170d9e8d6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:12.073034 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:12.250278 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddb2572cac446aa92a0bcf170d9e8d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.327817 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddb2572cac446aa92a0bcf170d9e8d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.328435 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddb2572cac446aa92a0bcf170d9e8d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.329039 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddb2572cac446aa92a0bcf170d9e8d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.336318 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddb2572cac446aa92a0bcf170d9e8d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.353767 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1452/2920] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:27:12 2022] DELETE /identity/v3/users/7ddb2572cac446aa92a0bcf170d9e8d6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:27:12.588109 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:12.596954 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9798f-f276-4b55-9de3-d6265464e8a8 service neutron] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:12.655238 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:12.655874 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:12.656152 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:12.656425 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] PATH_INFO: `/v3/projects/7cc199dd87e24511a8d267603b571456` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:12.657544 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc199dd87e24511a8d267603b571456)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:12.659468 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:12.763138 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc199dd87e24511a8d267603b571456', 'request_id': 'req-6d1f16a2-4fc0-4b89-8628-daba285fb330'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.765031 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:12.769412 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc199dd87e24511a8d267603b571456', 'request_id': 'req-6d1f16a2-4fc0-4b89-8628-daba285fb330'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.836649 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc199dd87e24511a8d267603b571456', 'request_id': 'req-6d1f16a2-4fc0-4b89-8628-daba285fb330'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.847219 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] The token cache is being invalidate because project 7cc199dd87e24511a8d267603b571456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:27:12.847769 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.848890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:12.850479 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1469/2921] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:27:12 2022] DELETE /identity/v3/projects/7cc199dd87e24511a8d267603b571456 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:27:12.854519 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:12.857124 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:12.869289 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1b5af-08aa-421a-a90c-321325140568 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:12.926523 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:12.927158 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:12.927465 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:12.927743 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] PATH_INFO: `/v3/users/6dcd84cc95cc4a899133c320ed7d4209` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:12.928871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcd84cc95cc4a899133c320ed7d4209)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:12.930777 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:13.118666 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcd84cc95cc4a899133c320ed7d4209'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.179773 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcd84cc95cc4a899133c320ed7d4209'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.180404 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcd84cc95cc4a899133c320ed7d4209'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.181041 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcd84cc95cc4a899133c320ed7d4209'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.190140 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b995dc7d-c4b5-48d8-ad5d-b6cc9d5a03d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcd84cc95cc4a899133c320ed7d4209'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.198733 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:13.208997 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1453/2922] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:27:12 2022] DELETE /identity/v3/users/6dcd84cc95cc4a899133c320ed7d4209 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:27:13.465549 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:13.467005 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:13.479969 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f16a2-4fc0-4b89-8628-daba285fb330 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:13.541353 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:13.541991 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:13.542276 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:13.542572 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] PATH_INFO: `/v3/projects/e6e686683db641109e4b1ed65cbbd3a1` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:13.543731 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e686683db641109e4b1ed65cbbd3a1)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:13.545633 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:13.669419 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e686683db641109e4b1ed65cbbd3a1', 'request_id': 'req-8b614c84-3e0d-4a9d-8c16-55b1dec66948'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.677183 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e686683db641109e4b1ed65cbbd3a1', 'request_id': 'req-8b614c84-3e0d-4a9d-8c16-55b1dec66948'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.762716 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e686683db641109e4b1ed65cbbd3a1', 'request_id': 'req-8b614c84-3e0d-4a9d-8c16-55b1dec66948'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.774042 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] The token cache is being invalidate because project e6e686683db641109e4b1ed65cbbd3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:27:13.774327 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.775213 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:13.776705 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1470/2923] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:27:13 2022] DELETE /identity/v3/projects/e6e686683db641109e4b1ed65cbbd3a1 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:27:23.322125 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:23.770868 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:38.029968 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:38.030505 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:38.030634 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:38.085383 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] MFA Rules not processed for user `4c062a52bd70482188def01c1ed57ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 10:27:38.086598 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:38.150336 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1454/2924] 10.222.0.62 () {58 vars in 967 bytes} [Tue Sep 13 10:27:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 10:27:38.190421 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:38.191871 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:38.282348 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b614c84-3e0d-4a9d-8c16-55b1dec66948 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:38.287509 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '174cca9647954ad1afdd4e6b8fba18f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:38.288111 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] REQUEST_METHOD: `GET` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:38.288391 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:38.288664 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:38.297317 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:38.298890 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:38.306735 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1471/2925] 10.222.0.62 () {60 vars in 1384 bytes} [Tue Sep 13 10:27:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 10:27:46.145099 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:46.146581 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:46.214506 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c948-1973-404b-a9c4-21ce48ec8199 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:46.219597 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:46.220233 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:46.220518 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:46.220793 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] PATH_INFO: `/v3/users/31a26b7651fd463393526fab64ccf1c7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:46.221924 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a26b7651fd463393526fab64ccf1c7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:46.223969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:46.421607 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a26b7651fd463393526fab64ccf1c7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:46.486601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a26b7651fd463393526fab64ccf1c7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:46.487249 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a26b7651fd463393526fab64ccf1c7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:46.487854 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a26b7651fd463393526fab64ccf1c7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:46.495175 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a26b7651fd463393526fab64ccf1c7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:46.503280 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:46.512652 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1455/2926] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:27:46 2022] DELETE /identity/v3/users/31a26b7651fd463393526fab64ccf1c7 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:27:46.786279 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:46.798499 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a8bf9-0adc-4b20-bc8d-3af1c8e97d91 service swift] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:46.858365 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:46.859010 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:46.859341 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:46.859648 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] PATH_INFO: `/v3/projects/d3610ef4e07548e1a88d76356d8170a6` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:46.860786 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3610ef4e07548e1a88d76356d8170a6)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:46.862708 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:46.972638 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3610ef4e07548e1a88d76356d8170a6', 'request_id': 'req-6c396d11-7c55-495d-ad5f-046152fdf541'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:46.978651 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3610ef4e07548e1a88d76356d8170a6', 'request_id': 'req-6c396d11-7c55-495d-ad5f-046152fdf541'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.052435 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3610ef4e07548e1a88d76356d8170a6', 'request_id': 'req-6c396d11-7c55-495d-ad5f-046152fdf541'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.059732 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:47.064140 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] The token cache is being invalidate because project d3610ef4e07548e1a88d76356d8170a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:27:47.064433 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.065085 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.066595 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1472/2927] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:27:46 2022] DELETE /identity/v3/projects/d3610ef4e07548e1a88d76356d8170a6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:27:47.070753 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:47.074587 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:47.086456 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf49862-5429-4188-bfc1-fa20da548bd1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:47.151943 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:47.152501 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:47.152741 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:47.152969 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] PATH_INFO: `/v3/users/bc8b8efb205d4001a9085ef1265bab12` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:47.154197 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8b8efb205d4001a9085ef1265bab12)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:47.156103 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:47.351813 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8b8efb205d4001a9085ef1265bab12'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.429766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8b8efb205d4001a9085ef1265bab12'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.430391 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8b8efb205d4001a9085ef1265bab12'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.431008 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8b8efb205d4001a9085ef1265bab12'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.439069 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8b8efb205d4001a9085ef1265bab12'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.456469 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1456/2928] 10.222.0.62 () {58 vars in 1224 bytes} [Tue Sep 13 10:27:47 2022] DELETE /identity/v3/users/bc8b8efb205d4001a9085ef1265bab12 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 10:27:47.694187 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Authenticating user token {{(pid=81736) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:27:47.695887 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.fernet_utils [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81736) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:27:47.709303 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396d11-7c55-495d-ad5f-046152fdf541 admin admin] Validating token access rules against request {{(pid=81736) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:27:47.766592 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47dfc0e354ea47fc9cdf40b73108b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60981a9be15c4b04b78b97afc4285aa0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81736) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:27:47.767264 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:27:47.767552 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:27:47.767823 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] PATH_INFO: `/v3/projects/3b07517680644824bf4a6e622adc4960` {{(pid=81736) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:27:47.768937 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b07517680644824bf4a6e622adc4960)` {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:27:47.770837 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] RBAC: Authorization granted {{(pid=81736) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:27:47.897459 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b07517680644824bf4a6e622adc4960', 'request_id': 'req-242f7174-8918-4ae0-8ee7-8f4ee9436c41'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.903689 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b07517680644824bf4a6e622adc4960', 'request_id': 'req-242f7174-8918-4ae0-8ee7-8f4ee9436c41'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.984091 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b07517680644824bf4a6e622adc4960', 'request_id': 'req-242f7174-8918-4ae0-8ee7-8f4ee9436c41'} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.994733 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] The token cache is being invalidate because project 3b07517680644824bf4a6e622adc4960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81736) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 10:27:47.995018 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.995670 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG keystone.notifications [None req-242f7174-8918-4ae0-8ee7-8f4ee9436c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81736) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 10:27:47.997146 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: [pid: 81736|app: 0|req: 1473/2929] 10.222.0.62 () {58 vars in 1233 bytes} [Tue Sep 13 10:27:47 2022] DELETE /identity/v3/projects/3b07517680644824bf4a6e622adc4960 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 10:27:57.452904 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:27:57.991561 dv-cinder-854274-4-iscsi devstack@keystone.service[81736]: DEBUG dbcounter [-] [81736] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81736) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 10:29:47.276252 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 10:29:47.279591 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:29:47.349077 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a9bbe-ef07-4483-b617-f0f72cacf492 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 10:29:47.408710 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '751cbd1e77824b83aa3c9c03c637fadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc4aca9e25c442aa9332d584d003a09a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 10:29:47.409321 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 10:29:47.409601 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 10:29:47.409871 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 10:29:47.411604 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 10:29:47.422577 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 10:29:47.424093 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d51578-d9ed-4b7f-b885-b70611a9e4ce service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 10:29:47.433766 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1457/2930] 10.222.0.62 () {60 vars in 1365 bytes} [Tue Sep 13 10:29:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 10:29:57.429612 dv-cinder-854274-4-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=58 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}